[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:39:26 dist-test-slave-5k54 krb5kdc[19186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:39:26 dist-test-slave-5k54 krb5kdc[19186](info): set up 2 sockets
Aug 08 23:39:26 dist-test-slave-5k54 krb5kdc[19186](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 23:39:27.900875 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.003s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:39:28 dist-test-slave-5k54 krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:39:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:39:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:39:28.087940 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:45561
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:45561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:28.435369 19200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:28.435837 19200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:28.436014 19200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:28.443157 19200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:39:28.443254 19200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:28.443348 19200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:28.443462 19200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:39:28.443534 19200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:39:28.453559 19200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:45561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:45561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:28.455966 19200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:28.457916 19200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:28.476502 19205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:28.477255 19206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:28.477303 19208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:28.478125 19200 server_base.cc:1034] running on GCE node
I20260808 23:39:28.479207 19200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:28.481323 19200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:28.482529 19200 hybrid_clock.cc:648] HybridClock initialized: now 1786232368482426 us; error 106 us; skew 500 ppm
I20260808 23:39:28.483023 19200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:28 dist-test-slave-5k54 krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232368, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:28.486591 19200 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:39:28.489154 19200 webserver.cc:460] Webserver started at http://127.18.186.190:45207/ using document root <none> and password file <none>
I20260808 23:39:28.490284 19200 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:28.490447 19200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:28.490926 19200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:28.494906 19200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "48cfca812b2b41e29fc71665edad3f71"
format_stamp: "Formatted at 2026-08-08 23:39:28 on dist-test-slave-5k54"
server_key: "515a16cffc3368883496ee5752e5ddf5"
server_key_iv: "25ae2a5a1fc822beb62a13220103182e"
server_key_version: "encryptionkey@0"
I20260808 23:39:28.495764 19200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "48cfca812b2b41e29fc71665edad3f71"
format_stamp: "Formatted at 2026-08-08 23:39:28 on dist-test-slave-5k54"
server_key: "515a16cffc3368883496ee5752e5ddf5"
server_key_iv: "25ae2a5a1fc822beb62a13220103182e"
server_key_version: "encryptionkey@0"
I20260808 23:39:28.503412 19200 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260808 23:39:28.509059 19215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:28.510932 19200 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:39:28.511201 19200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "48cfca812b2b41e29fc71665edad3f71"
format_stamp: "Formatted at 2026-08-08 23:39:28 on dist-test-slave-5k54"
server_key: "515a16cffc3368883496ee5752e5ddf5"
server_key_iv: "25ae2a5a1fc822beb62a13220103182e"
server_key_version: "encryptionkey@0"
I20260808 23:39:28.511484 19200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:28.533535 19200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:28.538308 19200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:28.538650 19200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:28.561524 19200 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:45561
I20260808 23:39:28.561565 19267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:45561 every 8 connection(s)
I20260808 23:39:28.563850 19200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:39:28.566524 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 19200
I20260808 23:39:28.566959 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:39:28.570521 19178 external_mini_cluster.cc:1428] Setting key 7b703ce5d61942a21ebcc47d78cff7df
I20260808 23:39:28.572607 19268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:28.589377 19268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71: Bootstrap starting.
I20260808 23:39:28.596100 19268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:28.597599 19268 log.cc:824] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:28.601686 19268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71: No bootstrap required, opened a new log
Aug 08 23:39:28 dist-test-slave-5k54 krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:28.612413 19268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48cfca812b2b41e29fc71665edad3f71" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45561 } }
I20260808 23:39:28.613018 19268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:28.613207 19268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48cfca812b2b41e29fc71665edad3f71, State: Initialized, Role: FOLLOWER
I20260808 23:39:28.614534 19268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48cfca812b2b41e29fc71665edad3f71" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45561 } }
I20260808 23:39:28.614856 19268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:39:28.615195 19268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:39:28.615408 19268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:28.618220 19268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48cfca812b2b41e29fc71665edad3f71" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45561 } }
I20260808 23:39:28.619370 19268 leader_election.cc:302] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48cfca812b2b41e29fc71665edad3f71; no voters:
I20260808 23:39:28.620347 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:28.578468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34716 (local address 127.18.186.190:45561)
0808 23:39:28.579630 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:28.579642 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:28.582970 (+ 3328us) server_negotiation.cc:407] Connection header received
0808 23:39:28.593585 (+ 10615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:28.593645 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:28.594431 (+ 786us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:28.595010 (+ 579us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:28.598269 (+ 3259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:28.600219 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:28.601956 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:28.603189 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:28.603489 (+ 300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:28.609168 (+ 5679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:28.609245 (+ 77us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:28.609293 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:28.609336 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:28.614010 (+ 4674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:28.614917 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:28.614924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:28.614929 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:28.615205 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:28.615811 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:28.615817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:28.615821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:28.616601 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:28.617088 (+ 487us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:28.618081 (+ 993us) server_negotiation.cc:299] Negotiation successful
0808 23:39:28.618484 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":359,"threads_started":1}
I20260808 23:39:28.620707 19268 leader_election.cc:288] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:39:28.620878 19273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:28.623148 19273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [term 1 LEADER]: Becoming Leader. State: Replica: 48cfca812b2b41e29fc71665edad3f71, State: Running, Role: LEADER
I20260808 23:39:28.624099 19268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:39:28.624146 19273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48cfca812b2b41e29fc71665edad3f71" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45561 } }
I20260808 23:39:28.629700 19275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48cfca812b2b41e29fc71665edad3f71. Latest consensus state: current_term: 1 leader_uuid: "48cfca812b2b41e29fc71665edad3f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48cfca812b2b41e29fc71665edad3f71" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45561 } } }
I20260808 23:39:28.630064 19275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:28.630719 19274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48cfca812b2b41e29fc71665edad3f71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48cfca812b2b41e29fc71665edad3f71" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45561 } } }
I20260808 23:39:28.631071 19274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:28.633090 19281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:39:28.640921 19281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:39:28.655100 19281 catalog_manager.cc:1347] Generated new cluster ID: 6c27457331d24d248229f0d90f62be91
I20260808 23:39:28.655256 19281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:39:28.688308 19281 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:39:28.690096 19281 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:39:28.699870 19281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48cfca812b2b41e29fc71665edad3f71: Generated new TSK 0
I20260808 23:39:28.700846 19281 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:39:28.748433 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45561
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:29.086789 19296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:29.087128 19296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:29.087294 19296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:29.094636 19296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:29.094787 19296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:29.094970 19296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:39:29.105042 19296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45561
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:29.106434 19296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:29.107869 19296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:29.122036 19303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:29.122071 19305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:29.122117 19302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:29.122931 19296 server_base.cc:1034] running on GCE node
I20260808 23:39:29.123626 19296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:29.124576 19296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:29.125828 19296 hybrid_clock.cc:648] HybridClock initialized: now 1786232369125745 us; error 55 us; skew 500 ppm
I20260808 23:39:29.126202 19296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:29 dist-test-slave-5k54 krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232369, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:29.129436 19296 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:39:29.130843 19296 webserver.cc:460] Webserver started at http://127.18.186.129:44015/ using document root <none> and password file <none>
I20260808 23:39:29.131668 19296 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:29.131843 19296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:29.132205 19296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:29.135161 19296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "d8a0290b0d404b5c815a1ed9c29f3cc9"
format_stamp: "Formatted at 2026-08-08 23:39:29 on dist-test-slave-5k54"
server_key: "bf81df9914954e323d91a680356f0cfd"
server_key_iv: "4f53231a3e829c70e14a961765f6b272"
server_key_version: "encryptionkey@0"
I20260808 23:39:29.136018 19296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "d8a0290b0d404b5c815a1ed9c29f3cc9"
format_stamp: "Formatted at 2026-08-08 23:39:29 on dist-test-slave-5k54"
server_key: "bf81df9914954e323d91a680356f0cfd"
server_key_iv: "4f53231a3e829c70e14a961765f6b272"
server_key_version: "encryptionkey@0"
I20260808 23:39:29.141093 19296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:39:29.144534 19312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:29.145892 19296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:39:29.146060 19296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "d8a0290b0d404b5c815a1ed9c29f3cc9"
format_stamp: "Formatted at 2026-08-08 23:39:29 on dist-test-slave-5k54"
server_key: "bf81df9914954e323d91a680356f0cfd"
server_key_iv: "4f53231a3e829c70e14a961765f6b272"
server_key_version: "encryptionkey@0"
I20260808 23:39:29.146258 19296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:29.179864 19296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:29.183815 19296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:29.184115 19296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:29.185257 19296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:29.186910 19296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:29.186985 19296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:29.187220 19296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:29.187278 19296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:29.223385 19296 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:34537
I20260808 23:39:29.223513 19428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:34537 every 8 connection(s)
I20260808 23:39:29.224802 19296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:39:29.227236 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 19296
I20260808 23:39:29.227523 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:39:29.228210 19178 external_mini_cluster.cc:1428] Setting key 95abf5b33ebf641817bb8caa1f4526d7
Aug 08 23:39:29 dist-test-slave-5k54 krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232369, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
I20260808 23:39:29.249218 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:29.228535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:49869 (local address 127.18.186.190:45561)
0808 23:39:29.229033 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:29.229039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:29.229931 (+ 892us) server_negotiation.cc:407] Connection header received
0808 23:39:29.232843 (+ 2912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:29.232848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:29.232912 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:29.233051 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:29.235169 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:29.235937 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:29.237712 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:29.238467 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:29.238627 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:29.242204 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:29.242214 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:29.242222 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:29.242268 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:29.246117 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:29.246812 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:29.246819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:29.246824 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:29.246983 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:29.247400 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:29.247406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:29.247411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:29.247843 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:29.248010 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:29.248728 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 23:39:29.248901 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":241,"threads_started":1}
I20260808 23:39:29.250253 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:29.229054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45561 (local address 127.18.186.129:49869)
0808 23:39:29.229783 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:29.229819 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:29.232531 (+ 2712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:29.233343 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:29.233359 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:29.234186 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:29.234981 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:29.235008 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:29.236103 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:29.236111 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:29.237559 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:29.237570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:29.238678 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:29.238683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:29.238869 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:29.239604 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:29.239630 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:29.241984 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:29.246254 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:29.246261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:29.246273 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:29.246687 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:29.247096 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:29.247099 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:29.247102 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:29.247279 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:29.247977 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:29.247983 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:29.248310 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 23:39:29.248630 (+ 320us) client_negotiation.cc:240] Negotiation successful
0808 23:39:29.248999 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":326,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
I20260808 23:39:29.252020 19429 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45561
I20260808 23:39:29.252530 19429 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:29.253603 19429 heartbeater.cc:511] Master 127.18.186.190:45561 requested a full tablet report, sending...
I20260808 23:39:29.258005 19232 ts_manager.cc:194] Registered new tserver with Master: d8a0290b0d404b5c815a1ed9c29f3cc9 (127.18.186.129:34537)
I20260808 23:39:29.260092 19232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:49869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:39:29.276178 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45561
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:29.614245 19436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:29.614562 19436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:29.614727 19436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:29.622712 19436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:29.622880 19436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:29.623059 19436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:39:29.633380 19436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45561
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:29.634675 19436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:29.636121 19436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:29.649250 19441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:29.649288 19442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:29.649328 19444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:29.650535 19436 server_base.cc:1034] running on GCE node
I20260808 23:39:29.651312 19436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:29.652326 19436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:29.653590 19436 hybrid_clock.cc:648] HybridClock initialized: now 1786232369653541 us; error 52 us; skew 500 ppm
I20260808 23:39:29.653924 19436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:29 dist-test-slave-5k54 krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232369, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:29.657195 19436 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:39:29.658689 19436 webserver.cc:460] Webserver started at http://127.18.186.130:45281/ using document root <none> and password file <none>
I20260808 23:39:29.659417 19436 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:29.659621 19436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:29.659961 19436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:29.663035 19436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "728d363f730d4a7e8e24e9c26a5a365d"
format_stamp: "Formatted at 2026-08-08 23:39:29 on dist-test-slave-5k54"
server_key: "5c0741435ef5151346fa35c9e4c31370"
server_key_iv: "cf65e656c011c1665d9452906651193b"
server_key_version: "encryptionkey@0"
I20260808 23:39:29.663843 19436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "728d363f730d4a7e8e24e9c26a5a365d"
format_stamp: "Formatted at 2026-08-08 23:39:29 on dist-test-slave-5k54"
server_key: "5c0741435ef5151346fa35c9e4c31370"
server_key_iv: "cf65e656c011c1665d9452906651193b"
server_key_version: "encryptionkey@0"
I20260808 23:39:29.668958 19436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:39:29.672502 19452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:29.673774 19436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:39:29.673969 19436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "728d363f730d4a7e8e24e9c26a5a365d"
format_stamp: "Formatted at 2026-08-08 23:39:29 on dist-test-slave-5k54"
server_key: "5c0741435ef5151346fa35c9e4c31370"
server_key_iv: "cf65e656c011c1665d9452906651193b"
server_key_version: "encryptionkey@0"
I20260808 23:39:29.674165 19436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:29.691144 19436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:29.695163 19436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:29.695477 19436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:29.696467 19436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:29.697804 19436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:29.697909 19436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:29.698064 19436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:29.698139 19436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:29.735610 19436 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:37543
I20260808 23:39:29.735669 19573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:37543 every 8 connection(s)
I20260808 23:39:29.736936 19436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:39:29.743342 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 19436
I20260808 23:39:29.743607 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:39:29.744282 19178 external_mini_cluster.cc:1428] Setting key 762d6b6974df3f396cd01fe3cee9395a
Aug 08 23:39:29 dist-test-slave-5k54 krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232369, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:29.761468 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:29.739498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:52985 (local address 127.18.186.190:45561)
0808 23:39:29.739668 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:29.739673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:29.740886 (+ 1213us) server_negotiation.cc:407] Connection header received
0808 23:39:29.742672 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:29.742677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:29.742750 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:29.742850 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:29.745315 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:29.746351 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:29.749289 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:29.749707 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:29.749801 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:29.754408 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:29.754413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:29.754417 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:29.754448 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:29.757223 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:29.758666 (+ 1443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:29.758671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:29.758674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:29.758849 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:29.759599 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:29.759603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:29.759605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:29.759910 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:29.760055 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:29.760973 (+ 918us) server_negotiation.cc:299] Negotiation successful
0808 23:39:29.761160 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:39:29.762850 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:29.739918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45561 (local address 127.18.186.130:52985)
0808 23:39:29.740716 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:29.740758 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:29.742414 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:29.743250 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:29.743263 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:29.744196 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:29.745103 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:29.745125 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:29.746557 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:29.746563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:29.749137 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:29.749158 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:29.749998 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:29.750016 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:29.750378 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:29.751302 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:29.751346 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:29.754175 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:29.757371 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:29.757441 (+ 70us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:29.757479 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:29.758356 (+ 877us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:29.758988 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:29.759011 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:29.759017 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:29.759307 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:29.760064 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:29.760075 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:29.760556 (+ 481us) client_negotiation.cc:769] Sending connection context
0808 23:39:29.760876 (+ 320us) client_negotiation.cc:240] Negotiation successful
0808 23:39:29.761484 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":360,"threads_started":1}
I20260808 23:39:29.764361 19574 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45561
I20260808 23:39:29.764703 19574 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:29.765669 19574 heartbeater.cc:511] Master 127.18.186.190:45561 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
I20260808 23:39:29.767525 19232 ts_manager.cc:194] Registered new tserver with Master: 728d363f730d4a7e8e24e9c26a5a365d (127.18.186.130:37543)
I20260808 23:39:29.768378 19232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:52985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:39:29.792495 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45561
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:30.126533 19579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:30.126802 19579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:30.126982 19579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:30.134359 19579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:30.134490 19579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:30.134686 19579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:39:30.144258 19579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45561
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:30.145581 19579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:30.146855 19579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:30.160185 19584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:30.160162 19585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:30.160305 19587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:30.161542 19579 server_base.cc:1034] running on GCE node
I20260808 23:39:30.162202 19579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:30.163141 19579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:30.164371 19579 hybrid_clock.cc:648] HybridClock initialized: now 1786232370164323 us; error 55 us; skew 500 ppm
I20260808 23:39:30.164696 19579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:30.167860 19579 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:39:30.169586 19579 webserver.cc:460] Webserver started at http://127.18.186.131:32927/ using document root <none> and password file <none>
I20260808 23:39:30.170298 19579 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:30.170450 19579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:30.170790 19579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:30.173924 19579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "d5913254e084411a84cb00faecae98a3"
format_stamp: "Formatted at 2026-08-08 23:39:30 on dist-test-slave-5k54"
server_key: "7c63973ea1267d8cfddb9fd73d7f33f9"
server_key_iv: "8ab122e775d5bdcff68481f2038a4d69"
server_key_version: "encryptionkey@0"
I20260808 23:39:30.174734 19579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "d5913254e084411a84cb00faecae98a3"
format_stamp: "Formatted at 2026-08-08 23:39:30 on dist-test-slave-5k54"
server_key: "7c63973ea1267d8cfddb9fd73d7f33f9"
server_key_iv: "8ab122e775d5bdcff68481f2038a4d69"
server_key_version: "encryptionkey@0"
I20260808 23:39:30.180332 19579 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 23:39:30.183853 19594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:30.185209 19579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 23:39:30.185385 19579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "d5913254e084411a84cb00faecae98a3"
format_stamp: "Formatted at 2026-08-08 23:39:30 on dist-test-slave-5k54"
server_key: "7c63973ea1267d8cfddb9fd73d7f33f9"
server_key_iv: "8ab122e775d5bdcff68481f2038a4d69"
server_key_version: "encryptionkey@0"
I20260808 23:39:30.185592 19579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:30.208429 19579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:30.212453 19579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:30.212752 19579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:30.213775 19579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:30.215080 19579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:30.215186 19579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:30.215342 19579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:30.215436 19579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:30.253328 19579 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:36691
I20260808 23:39:30.253376 19714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:36691 every 8 connection(s)
I20260808 23:39:30.254649 19579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:39:30.259856 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 19579
I20260808 23:39:30.260093 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:39:30.260646 19178 external_mini_cluster.cc:1428] Setting key 5649bd148b0c57a6d7f1b5fd175519d3
I20260808 23:39:30.263367 19429 heartbeater.cc:503] Master 127.18.186.190:45561 was elected leader, sending a full tablet report...
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:30.275483 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:30.257192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:60483 (local address 127.18.186.190:45561)
0808 23:39:30.257422 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:30.257429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:30.258713 (+ 1284us) server_negotiation.cc:407] Connection header received
0808 23:39:30.261188 (+ 2475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:30.261193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:30.261283 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:30.261385 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:30.263362 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.264470 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.266246 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.266663 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.266799 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:30.270282 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:30.270286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:30.270290 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:30.270321 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:30.272895 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.273559 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.273562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.273565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.273693 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.274069 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.274072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.274074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.274307 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:30.274421 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:30.275030 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 23:39:30.275180 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:39:30.276602 19717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:30.257689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45561 (local address 127.18.186.131:60483)
0808 23:39:30.258440 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:30.258484 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:30.260862 (+ 2378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:30.261688 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:30.261696 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:30.262284 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:30.263139 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:30.263169 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.264636 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.264641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:30.266103 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:30.266114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.266900 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.266908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:30.267144 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:30.267857 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:30.267879 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:30.270076 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:30.273019 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:30.273026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:30.273037 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:30.273462 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:30.273786 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:30.273789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:30.273791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:30.273977 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:30.274416 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:30.274421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:30.274716 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 23:39:30.274940 (+ 224us) client_negotiation.cc:240] Negotiation successful
0808 23:39:30.275298 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":318,"threads_started":1}
I20260808 23:39:30.278110 19715 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45561
I20260808 23:39:30.278416 19715 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:30.279147 19715 heartbeater.cc:511] Master 127.18.186.190:45561 requested a full tablet report, sending...
I20260808 23:39:30.280535 19232 ts_manager.cc:194] Registered new tserver with Master: d5913254e084411a84cb00faecae98a3 (127.18.186.131:36691)
I20260808 23:39:30.281318 19232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:60483
I20260808 23:39:30.290198 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:39:30.338642 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:30.323015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.18.186.129:34537)
0808 23:39:30.323722 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:30.323729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:30.323757 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:39:30.323816 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:30.323823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:30.324040 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:30.324168 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:30.325848 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.326864 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.328198 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.328753 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.328857 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:30.333286 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:30.333297 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:30.333311 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:30.333342 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:30.336160 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.336748 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.336756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.336758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.336875 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.337198 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.337201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.337203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.337572 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:30.337797 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:30.338061 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 23:39:30.338245 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":404,"threads_started":1}
W20260808 23:39:30.340045 19343 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59186
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:39:30.387383 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:30.372914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59194 (local address 127.18.186.129:34537)
0808 23:39:30.373162 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:30.373170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:30.373496 (+ 326us) server_negotiation.cc:407] Connection header received
0808 23:39:30.373729 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:30.373733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:30.373828 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:30.373942 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:30.375299 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.376066 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.377305 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.377784 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.377941 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:30.381970 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:30.381974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:30.381978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:30.382012 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:30.384735 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.385480 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.385487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.385491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.385625 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.385996 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.386002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.386006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.386265 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:30.386519 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:30.386832 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 23:39:30.387061 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:39:30.389396 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 19296
I20260808 23:39:30.402245 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:30.390601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59198 (local address 127.18.186.129:34537)
0808 23:39:30.390757 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:30.390761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:30.391001 (+ 240us) server_negotiation.cc:407] Connection header received
0808 23:39:30.391110 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:30.391114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:30.391165 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:30.391242 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:30.392456 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.393122 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.394328 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.394769 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.394945 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:30.397312 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:30.397317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:30.397320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:30.397349 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:30.399977 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.400668 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.400671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.400673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.400798 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.401120 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.401123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.401125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.401355 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:30.401540 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:30.401830 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 23:39:30.401990 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:39:30.403087 19424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:30.494576 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 19296
I20260808 23:39:30.513255 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 19436
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:39:30.530618 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:30.514330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.18.186.130:37543)
0808 23:39:30.514909 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:30.514918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:30.514945 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:39:30.515014 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:30.515021 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:30.515242 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:30.515399 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:30.516828 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.518172 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.519380 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.520051 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.520157 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:30.524480 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:30.524491 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:30.524506 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:30.524541 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:30.527910 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.528575 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.528582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.528585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.528718 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.529121 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.529125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.529127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.529465 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:30.529690 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:30.529994 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 23:39:30.530195 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":308,"threads_started":1}
I20260808 23:39:30.532217 19569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:30.617866 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 19436
I20260808 23:39:30.636559 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 19579
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:39:30.652688 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:30.637498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.18.186.131:36691)
0808 23:39:30.638116 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:30.638123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:30.638152 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:39:30.638215 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:30.638222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:30.638440 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:30.638619 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:30.639980 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.641035 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.642304 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:30.642883 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:30.643033 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:30.647350 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:30.647361 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:30.647374 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:30.647407 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:30.650110 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.650734 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.650741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.650743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.650912 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:30.651226 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:30.651229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:30.651231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:30.651720 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:30.651977 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:30.652200 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 23:39:30.652399 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":270,"threads_started":1}
I20260808 23:39:30.653995 19710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:30.737026 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 19579
I20260808 23:39:30.755645 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 19200
I20260808 23:39:30.756858 19263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:30.859804 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 19200
2026-08-08T23:39:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4116 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19738](info): set up 2 sockets
Aug 08 23:39:30 dist-test-slave-5k54 krb5kdc[19738](info): commencing operation
krb5kdc: starting...
W20260808 23:39:31.986590 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:39:32 dist-test-slave-5k54 krb5kdc[19738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:39:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:39:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:39:32.177326 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:37939
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:43853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:37939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:32.490451 19751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:32.490731 19751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:32.490820 19751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:32.497797 19751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:39:32.497889 19751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:32.497936 19751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:32.497991 19751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:39:32.498035 19751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:39:32.507550 19751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:37939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:37939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:32.508754 19751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:32.510004 19751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:32.521529 19756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:32.521518 19760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:32.521545 19758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:32.522931 19751 server_base.cc:1034] running on GCE node
I20260808 23:39:32.523726 19751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:32.525075 19751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:32.526276 19751 hybrid_clock.cc:648] HybridClock initialized: now 1786232372526237 us; error 25 us; skew 500 ppm
I20260808 23:39:32.526576 19751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:32 dist-test-slave-5k54 krb5kdc[19738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232372, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:32.534199 19751 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:39:32.535831 19751 webserver.cc:460] Webserver started at http://127.18.186.190:34061/ using document root <none> and password file <none>
I20260808 23:39:32.536520 19751 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:32.536665 19751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:32.536988 19751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:32.539956 19751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "04fccec3f51e4f8fa168c1867250466a"
format_stamp: "Formatted at 2026-08-08 23:39:32 on dist-test-slave-5k54"
server_key: "92e75eac57cc07314e772e3e745deb13"
server_key_iv: "696fd9892e56dc816debf13182c19320"
server_key_version: "encryptionkey@0"
I20260808 23:39:32.540709 19751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "04fccec3f51e4f8fa168c1867250466a"
format_stamp: "Formatted at 2026-08-08 23:39:32 on dist-test-slave-5k54"
server_key: "92e75eac57cc07314e772e3e745deb13"
server_key_iv: "696fd9892e56dc816debf13182c19320"
server_key_version: "encryptionkey@0"
I20260808 23:39:32.546222 19751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:39:32.549629 19767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:32.550936 19751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:39:32.551137 19751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "04fccec3f51e4f8fa168c1867250466a"
format_stamp: "Formatted at 2026-08-08 23:39:32 on dist-test-slave-5k54"
server_key: "92e75eac57cc07314e772e3e745deb13"
server_key_iv: "696fd9892e56dc816debf13182c19320"
server_key_version: "encryptionkey@0"
I20260808 23:39:32.551340 19751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:32.565764 19751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:32.570076 19751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:32.570372 19751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:32.588749 19751 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:37939
I20260808 23:39:32.588768 19828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:37939 every 8 connection(s)
I20260808 23:39:32.590091 19751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:39:32.593811 19829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:32.594142 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 19751
I20260808 23:39:32.594379 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:39:32.595062 19178 external_mini_cluster.cc:1428] Setting key b8cd74867de62d1b645d04145e77c139
I20260808 23:39:32.607687 19829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a: Bootstrap starting.
Aug 08 23:39:32 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:32.611289 19829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:32.612558 19829 log.cc:824] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:32.615307 19829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a: No bootstrap required, opened a new log
I20260808 23:39:32.618148 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:32.597067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.18.186.190:37939)
0808 23:39:32.597926 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:32.597936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:32.597974 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:39:32.599830 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:32.599855 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:32.600235 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:32.600583 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:32.602387 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:32.603744 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:32.605584 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:32.606243 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:32.606366 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:32.610964 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:32.610971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:32.610986 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:32.611020 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:32.614133 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:32.614779 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:32.614787 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:32.614795 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:32.614966 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:32.615346 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:32.615351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:32.615355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:32.615959 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:32.616257 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:32.616759 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 23:39:32.617166 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":382,"threads_started":1}
I20260808 23:39:32.623214 19829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fccec3f51e4f8fa168c1867250466a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37939 } }
I20260808 23:39:32.623684 19829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:32.623804 19829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04fccec3f51e4f8fa168c1867250466a, State: Initialized, Role: FOLLOWER
I20260808 23:39:32.624393 19829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fccec3f51e4f8fa168c1867250466a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37939 } }
I20260808 23:39:32.624627 19829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:39:32.624752 19829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:39:32.625020 19829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:32.627113 19829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fccec3f51e4f8fa168c1867250466a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37939 } }
I20260808 23:39:32.627638 19829 leader_election.cc:302] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04fccec3f51e4f8fa168c1867250466a; no voters:
I20260808 23:39:32.628160 19829 leader_election.cc:288] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:39:32.628260 19834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:32.629704 19834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [term 1 LEADER]: Becoming Leader. State: Replica: 04fccec3f51e4f8fa168c1867250466a, State: Running, Role: LEADER
I20260808 23:39:32.630199 19829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:39:32.630216 19834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fccec3f51e4f8fa168c1867250466a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37939 } }
I20260808 23:39:32.633177 19835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04fccec3f51e4f8fa168c1867250466a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fccec3f51e4f8fa168c1867250466a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37939 } } }
I20260808 23:39:32.633205 19836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04fccec3f51e4f8fa168c1867250466a. Latest consensus state: current_term: 1 leader_uuid: "04fccec3f51e4f8fa168c1867250466a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fccec3f51e4f8fa168c1867250466a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37939 } } }
I20260808 23:39:32.633682 19835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:32.633739 19836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:32.639097 19842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:39:32.643333 19842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:39:32.651626 19842 catalog_manager.cc:1347] Generated new cluster ID: 1e6157a09a2e4b398521923049ca3ef8
I20260808 23:39:32.651749 19842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:39:32.680768 19842 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:39:32.682193 19842 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:39:32.698359 19842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04fccec3f51e4f8fa168c1867250466a: Generated new TSK 0
I20260808 23:39:32.699218 19842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:39:32.758496 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37939
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:43853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:33.088802 19857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:33.089072 19857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:33.089238 19857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:33.096357 19857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:33.096458 19857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:33.096671 19857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:39:33.106458 19857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:33.107896 19857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:33.109236 19857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:33.123327 19865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:33.123469 19862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:33.123415 19863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:33.124106 19857 server_base.cc:1034] running on GCE node
I20260808 23:39:33.124711 19857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:33.125643 19857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:33.126868 19857 hybrid_clock.cc:648] HybridClock initialized: now 1786232373126829 us; error 53 us; skew 500 ppm
I20260808 23:39:33.127228 19857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:33 dist-test-slave-5k54 krb5kdc[19738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232373, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:33.130470 19857 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:39:33.132061 19857 webserver.cc:460] Webserver started at http://127.18.186.129:39723/ using document root <none> and password file <none>
I20260808 23:39:33.132823 19857 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:33.132928 19857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:33.133212 19857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:33.136086 19857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "53ac9e30c86b4be39354d7de52e67431"
format_stamp: "Formatted at 2026-08-08 23:39:33 on dist-test-slave-5k54"
server_key: "e1b988c4006b0e9cbfa033aca2e9671d"
server_key_iv: "c48ba3681ceb91c594480191ff58e590"
server_key_version: "encryptionkey@0"
I20260808 23:39:33.137042 19857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "53ac9e30c86b4be39354d7de52e67431"
format_stamp: "Formatted at 2026-08-08 23:39:33 on dist-test-slave-5k54"
server_key: "e1b988c4006b0e9cbfa033aca2e9671d"
server_key_iv: "c48ba3681ceb91c594480191ff58e590"
server_key_version: "encryptionkey@0"
I20260808 23:39:33.142263 19857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:39:33.145599 19873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:33.146849 19857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:39:33.147025 19857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "53ac9e30c86b4be39354d7de52e67431"
format_stamp: "Formatted at 2026-08-08 23:39:33 on dist-test-slave-5k54"
server_key: "e1b988c4006b0e9cbfa033aca2e9671d"
server_key_iv: "c48ba3681ceb91c594480191ff58e590"
server_key_version: "encryptionkey@0"
I20260808 23:39:33.147226 19857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:33.166409 19857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:33.170301 19857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:33.170616 19857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:33.171593 19857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:33.172871 19857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:33.172976 19857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:33.173132 19857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:33.173206 19857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:33.207118 19857 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:40499
I20260808 23:39:33.207218 19994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:40499 every 8 connection(s)
I20260808 23:39:33.208523 19857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:39:33.209246 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 19857
I20260808 23:39:33.209494 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:39:33.210320 19178 external_mini_cluster.cc:1428] Setting key cb93a2ee2a4124b6958a198688c34d37
Aug 08 23:39:33 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232373, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
I20260808 23:39:33.231841 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:33.211574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:47587 (local address 127.18.186.190:37939)
0808 23:39:33.212027 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:33.212033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:33.213223 (+ 1190us) server_negotiation.cc:407] Connection header received
0808 23:39:33.215098 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:33.215106 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:33.215207 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:33.215330 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:33.216904 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:33.218057 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:33.220733 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:33.221144 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:33.221236 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:33.225735 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:33.225740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:33.225744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:33.225775 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:33.228429 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:33.229194 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:33.229199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:33.229201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:33.229317 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:33.229752 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:33.229757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:33.229760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:33.230094 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:33.230303 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:33.231163 (+ 860us) server_negotiation.cc:299] Negotiation successful
0808 23:39:33.231400 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":211,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
I20260808 23:39:33.232962 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:33.212121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37939 (local address 127.18.186.129:47587)
0808 23:39:33.213039 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:33.213090 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:33.214819 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:33.215595 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:33.215603 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:33.216154 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:33.216716 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:33.216734 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:33.218217 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:33.218223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:33.220567 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:33.220586 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:33.221381 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:33.221390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:33.221726 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:33.222622 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:33.222664 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:33.225488 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:33.228583 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:33.228592 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:33.228609 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:33.229049 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:33.229435 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:33.229440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:33.229443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:33.229635 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:33.230274 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:33.230282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:33.230742 (+ 460us) client_negotiation.cc:769] Sending connection context
0808 23:39:33.231046 (+ 304us) client_negotiation.cc:240] Negotiation successful
0808 23:39:33.231615 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":370,"threads_started":1}
I20260808 23:39:33.234699 19995 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37939
I20260808 23:39:33.235008 19995 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:33.235828 19995 heartbeater.cc:511] Master 127.18.186.190:37939 requested a full tablet report, sending...
I20260808 23:39:33.237947 19788 ts_manager.cc:194] Registered new tserver with Master: 53ac9e30c86b4be39354d7de52e67431 (127.18.186.129:40499)
I20260808 23:39:33.239359 19788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:47587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:39:33.258311 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37939
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:43853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:33.588207 20001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:33.588538 20001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:33.588714 20001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:33.596066 20001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:33.596217 20001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:33.596413 20001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:39:33.606184 20001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:33.607517 20001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:33.608866 20001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:33.621810 20006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:33.621817 20007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:33.621838 20009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:33.622684 20001 server_base.cc:1034] running on GCE node
I20260808 23:39:33.623314 20001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:33.624351 20001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:33.625548 20001 hybrid_clock.cc:648] HybridClock initialized: now 1786232373625487 us; error 53 us; skew 500 ppm
I20260808 23:39:33.625885 20001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:33 dist-test-slave-5k54 krb5kdc[19738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232373, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:33.629022 20001 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:39:33.630530 20001 webserver.cc:460] Webserver started at http://127.18.186.130:41625/ using document root <none> and password file <none>
I20260808 23:39:33.631263 20001 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:33.631413 20001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:33.631788 20001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:33.634676 20001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "ebbc22a83f914374bda0102cc918b91d"
format_stamp: "Formatted at 2026-08-08 23:39:33 on dist-test-slave-5k54"
server_key: "f187cd9b9c2dbbbfc3167feacdcbcbae"
server_key_iv: "42b955e367353c57356f75dbc2c9c34f"
server_key_version: "encryptionkey@0"
I20260808 23:39:33.635531 20001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "ebbc22a83f914374bda0102cc918b91d"
format_stamp: "Formatted at 2026-08-08 23:39:33 on dist-test-slave-5k54"
server_key: "f187cd9b9c2dbbbfc3167feacdcbcbae"
server_key_iv: "42b955e367353c57356f75dbc2c9c34f"
server_key_version: "encryptionkey@0"
I20260808 23:39:33.640690 20001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:39:33.644106 20016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:33.645428 20001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:39:33.645594 20001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "ebbc22a83f914374bda0102cc918b91d"
format_stamp: "Formatted at 2026-08-08 23:39:33 on dist-test-slave-5k54"
server_key: "f187cd9b9c2dbbbfc3167feacdcbcbae"
server_key_iv: "42b955e367353c57356f75dbc2c9c34f"
server_key_version: "encryptionkey@0"
I20260808 23:39:33.645815 20001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:33.675864 20001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:33.679873 20001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:33.680176 20001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:33.681176 20001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:33.682503 20001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:33.682614 20001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:33.682775 20001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:33.682861 20001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:33.717674 20001 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:38177
I20260808 23:39:33.717715 20132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:38177 every 8 connection(s)
I20260808 23:39:33.719084 20001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:39:33.727759 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 20001
I20260808 23:39:33.727972 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:39:33.728564 19178 external_mini_cluster.cc:1428] Setting key dbade7b1b6079195e93c55c0e7e1e184
Aug 08 23:39:33 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232373, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:33.741245 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:33.721725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:47945 (local address 127.18.186.190:37939)
0808 23:39:33.721930 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:33.721936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:33.723175 (+ 1239us) server_negotiation.cc:407] Connection header received
0808 23:39:33.725067 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:33.725072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:33.725138 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:33.725228 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:33.726656 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:33.727348 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:33.729660 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:33.730303 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:33.730505 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:33.734727 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:33.734733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:33.734736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:33.734767 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:33.737941 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:33.738623 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:33.738630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:33.738637 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:33.738787 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:33.739209 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:33.739217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:33.739224 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:33.739626 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:33.739781 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:33.740530 (+ 749us) server_negotiation.cc:299] Negotiation successful
0808 23:39:33.740792 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:39:33.741879 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:33.722302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37939 (local address 127.18.186.130:47945)
0808 23:39:33.723017 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:33.723053 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:33.724824 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:33.725430 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:33.725438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:33.725955 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:33.726515 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:33.726531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:33.727578 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:33.727586 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:33.729497 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:33.729511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:33.730530 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:33.730539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:33.730849 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:33.731569 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:33.731590 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:33.734504 (+ 2914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:33.738068 (+ 3564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:33.738074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:33.738086 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:33.738506 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:33.738918 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:33.738921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:33.738923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:33.739103 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:33.739855 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:33.739861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:33.740169 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 23:39:33.740432 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 23:39:33.740795 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":276,"threads_started":1}
I20260808 23:39:33.743319 20133 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37939
I20260808 23:39:33.743646 20133 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:33.744462 20133 heartbeater.cc:511] Master 127.18.186.190:37939 requested a full tablet report, sending...
I20260808 23:39:33.745977 19788 ts_manager.cc:194] Registered new tserver with Master: ebbc22a83f914374bda0102cc918b91d (127.18.186.130:38177)
I20260808 23:39:33.746795 19788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:47945
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:39:33.774432 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37939
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:43853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:34.098049 20139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:34.098318 20139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:34.098416 20139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:34.105614 20139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:34.105732 20139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:34.105850 20139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:39:34.115527 20139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:34.116777 20139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:34.118070 20139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:34.131481 20145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:34.131527 20147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:34.131526 20144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:34.132620 20139 server_base.cc:1034] running on GCE node
I20260808 23:39:34.133221 20139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:34.134151 20139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:34.135349 20139 hybrid_clock.cc:648] HybridClock initialized: now 1786232374135294 us; error 45 us; skew 500 ppm
I20260808 23:39:34.135716 20139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:34 dist-test-slave-5k54 krb5kdc[19738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232374, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:34.139002 20139 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:39:34.140524 20139 webserver.cc:460] Webserver started at http://127.18.186.131:46247/ using document root <none> and password file <none>
I20260808 23:39:34.141209 20139 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:34.141357 20139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:34.141674 20139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:34.144577 20139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "eb80a9f490ae40fc90079c986574ee78"
format_stamp: "Formatted at 2026-08-08 23:39:34 on dist-test-slave-5k54"
server_key: "9d3591811ffa67a03d72deaa13191c79"
server_key_iv: "cfee2be12e66497d3f343351d4edeeb7"
server_key_version: "encryptionkey@0"
I20260808 23:39:34.145382 20139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "eb80a9f490ae40fc90079c986574ee78"
format_stamp: "Formatted at 2026-08-08 23:39:34 on dist-test-slave-5k54"
server_key: "9d3591811ffa67a03d72deaa13191c79"
server_key_iv: "cfee2be12e66497d3f343351d4edeeb7"
server_key_version: "encryptionkey@0"
I20260808 23:39:34.150591 20139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:39:34.153991 20154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:34.155409 20139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:39:34.155622 20139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "eb80a9f490ae40fc90079c986574ee78"
format_stamp: "Formatted at 2026-08-08 23:39:34 on dist-test-slave-5k54"
server_key: "9d3591811ffa67a03d72deaa13191c79"
server_key_iv: "cfee2be12e66497d3f343351d4edeeb7"
server_key_version: "encryptionkey@0"
I20260808 23:39:34.155798 20139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:34.182111 20139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:34.186038 20139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:34.186285 20139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:34.187189 20139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:34.188477 20139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:34.188550 20139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:34.188633 20139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:34.188678 20139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:34.222913 20139 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:46253
I20260808 23:39:34.222952 20271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:46253 every 8 connection(s)
I20260808 23:39:34.224262 20139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:39:34.232127 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 20139
I20260808 23:39:34.232337 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:39:34.232764 19178 external_mini_cluster.cc:1428] Setting key b71fbbab35d04d8a1758f48039333653
Aug 08 23:39:34 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232374, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:34.242280 19995 heartbeater.cc:503] Master 127.18.186.190:37939 was elected leader, sending a full tablet report...
I20260808 23:39:34.245319 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.226922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:41183 (local address 127.18.186.190:37939)
0808 23:39:34.227102 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.227106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.228307 (+ 1201us) server_negotiation.cc:407] Connection header received
0808 23:39:34.230122 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.230126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.230208 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.230305 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:34.232031 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.232915 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.234851 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.235288 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.235389 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.239018 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:34.239022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:34.239026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:34.239057 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:34.242033 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.242968 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.242973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.242977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.243144 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.243681 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.243684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.243686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.244053 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:34.244215 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.244876 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.245057 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:39:34.246397 20274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.227394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37939 (local address 127.18.186.131:41183)
0808 23:39:34.228146 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:34.228183 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:34.229825 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:34.230535 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:34.230542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:34.231100 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:34.231841 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.231869 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.233070 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.233076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.234718 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.234734 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.235544 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.235553 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.235909 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.236619 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:34.236643 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:34.238841 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:34.242178 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:34.242185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:34.242197 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:34.242810 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:34.243264 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:34.243268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:34.243272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:34.243567 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:34.244196 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:34.244202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:34.244518 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 23:39:34.244754 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 23:39:34.245173 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":278,"threads_started":1}
I20260808 23:39:34.247931 20272 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37939
I20260808 23:39:34.248281 20272 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:34.249022 20272 heartbeater.cc:511] Master 127.18.186.190:37939 requested a full tablet report, sending...
I20260808 23:39:34.250486 19788 ts_manager.cc:194] Registered new tserver with Master: eb80a9f490ae40fc90079c986574ee78 (127.18.186.131:46253)
I20260808 23:39:34.251384 19788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:41183
I20260808 23:39:34.260500 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:39:34.279902 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.267576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.18.186.190:37939)
0808 23:39:34.267724 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.267729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.268283 (+ 554us) server_negotiation.cc:407] Connection header received
0808 23:39:34.268416 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.268420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.268483 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.268563 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:34.270068 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.270790 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.272037 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.272450 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.272543 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.274780 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:34.274785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:34.274788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:34.274815 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:34.277362 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.278002 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.278006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.278008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.278133 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.278608 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.278612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.278614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.278923 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:34.279126 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.279389 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.279565 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 23:39:34.287967 19788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:39:34.295917 19788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:34.337095 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.310725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57726 (local address 127.18.186.130:38177)
0808 23:39:34.328993 (+ 18268us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.329005 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.329032 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:39:34.329129 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.329137 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.329345 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.329560 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:34.330429 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.332050 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.334136 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.335729 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.335910 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.336095 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.336470 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.336659 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18087,"thread_start_us":18425,"threads_started":1}
I20260808 23:39:34.337236 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.310633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:38177 (local address 127.0.0.1:57726)
0808 23:39:34.326314 (+ 15681us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:34.326382 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:34.326684 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:34.329586 (+ 2902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:34.329591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:34.329624 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:34.330238 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.330251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.332250 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.332258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.333978 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.333990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.335937 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.335945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.336139 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.336196 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 23:39:34.336346 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 23:39:34.336570 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15345,"mutex_wait_us":103,"thread_start_us":17747,"threads_started":1}
I20260808 23:39:34.337461 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.310780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:40499 (local address 127.0.0.1:49592)
0808 23:39:34.328113 (+ 17333us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:34.328138 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:34.328291 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:34.328933 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:34.328944 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:34.328973 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:34.329352 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.329368 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.331705 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.331710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.334171 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.334186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.336367 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.336383 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.336635 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.336671 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:39:34.336785 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 23:39:34.336933 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17132,"mutex_wait_us":27,"thread_start_us":17151,"threads_started":1}
I20260808 23:39:34.338315 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.311492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49592 (local address 127.18.186.129:40499)
0808 23:39:34.321621 (+ 10129us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.321634 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.328253 (+ 6619us) server_negotiation.cc:407] Connection header received
0808 23:39:34.328460 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.328468 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.328712 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.328918 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:34.329533 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.331540 (+ 2007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.334336 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.335740 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.336004 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.336215 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.337691 (+ 1476us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.337931 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9903,"thread_start_us":9660,"threads_started":1}
I20260808 23:39:34.338651 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.311700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.18.186.131:46253)
0808 23:39:34.320423 (+ 8723us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.320437 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.328512 (+ 8075us) server_negotiation.cc:407] Connection header received
0808 23:39:34.328631 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.328641 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.328899 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.329101 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:34.330210 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.332047 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.335070 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.336567 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.336761 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.337057 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.338087 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.338267 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8486,"thread_start_us":8268,"threads_started":1}
I20260808 23:39:34.339983 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.311061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:46253 (local address 127.0.0.1:41236)
0808 23:39:34.328236 (+ 17175us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:34.328256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:34.328379 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:34.329420 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:34.329424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:34.329453 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:34.330009 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.330026 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.332199 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.332205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.334878 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.334901 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.337320 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.337328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.337856 (+ 528us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.337896 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 23:39:34.338017 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 23:39:34.338155 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17037,"mutex_wait_us":33,"thread_start_us":15882,"threads_started":1}
I20260808 23:39:34.342597 20065 tablet_service.cc:1467] Processing CreateTablet for tablet 92424139fd2041b39d2081c532bdf644 (DEFAULT_TABLE table=test-table [id=f904c97f8fe7473c8a71d48af53a74b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:34.342867 19925 tablet_service.cc:1467] Processing CreateTablet for tablet 92424139fd2041b39d2081c532bdf644 (DEFAULT_TABLE table=test-table [id=f904c97f8fe7473c8a71d48af53a74b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:34.344327 20065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92424139fd2041b39d2081c532bdf644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:34.344036 20204 tablet_service.cc:1467] Processing CreateTablet for tablet 92424139fd2041b39d2081c532bdf644 (DEFAULT_TABLE table=test-table [id=f904c97f8fe7473c8a71d48af53a74b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:34.344754 19925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92424139fd2041b39d2081c532bdf644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:34.345599 20204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92424139fd2041b39d2081c532bdf644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:34.361510 20293 tablet_bootstrap.cc:492] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78: Bootstrap starting.
I20260808 23:39:34.364327 20291 tablet_bootstrap.cc:492] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d: Bootstrap starting.
I20260808 23:39:34.366194 20293 tablet_bootstrap.cc:654] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:34.366931 20292 tablet_bootstrap.cc:492] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431: Bootstrap starting.
I20260808 23:39:34.367769 20293 log.cc:824] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:34.368847 20291 tablet_bootstrap.cc:654] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:34.370373 20291 log.cc:824] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:34.371133 20293 tablet_bootstrap.cc:492] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78: No bootstrap required, opened a new log
I20260808 23:39:34.371470 20293 ts_tablet_manager.cc:1397] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260808 23:39:34.371816 20292 tablet_bootstrap.cc:654] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:34.372838 20292 log.cc:824] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:34.373826 20291 tablet_bootstrap.cc:492] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d: No bootstrap required, opened a new log
I20260808 23:39:34.374109 20291 ts_tablet_manager.cc:1397] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260808 23:39:34.375619 20292 tablet_bootstrap.cc:492] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431: No bootstrap required, opened a new log
I20260808 23:39:34.375887 20292 ts_tablet_manager.cc:1397] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 23:39:34.383987 20293 raft_consensus.cc:348] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.384512 20293 raft_consensus.cc:374] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:34.384249 20292 raft_consensus.cc:348] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.384666 20292 raft_consensus.cc:374] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:34.384675 20293 raft_consensus.cc:729] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb80a9f490ae40fc90079c986574ee78, State: Initialized, Role: FOLLOWER
I20260808 23:39:34.384830 20292 raft_consensus.cc:729] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ac9e30c86b4be39354d7de52e67431, State: Initialized, Role: FOLLOWER
I20260808 23:39:34.385489 20292 consensus_queue.cc:260] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.385517 20293 consensus_queue.cc:260] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.386206 20291 raft_consensus.cc:348] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.386844 20291 raft_consensus.cc:374] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:34.387005 20291 raft_consensus.cc:729] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebbc22a83f914374bda0102cc918b91d, State: Initialized, Role: FOLLOWER
I20260808 23:39:34.387610 20292 ts_tablet_manager.cc:1428] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 23:39:34.387948 20291 consensus_queue.cc:260] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.389338 20272 heartbeater.cc:503] Master 127.18.186.190:37939 was elected leader, sending a full tablet report...
I20260808 23:39:34.389350 20133 heartbeater.cc:503] Master 127.18.186.190:37939 was elected leader, sending a full tablet report...
I20260808 23:39:34.390143 20291 ts_tablet_manager.cc:1428] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
I20260808 23:39:34.391817 20293 ts_tablet_manager.cc:1428] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78: Time spent starting tablet: real 0.020s user 0.013s sys 0.005s
W20260808 23:39:34.461223 19996 tablet.cc:2369] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:39:34.471243 20134 tablet.cc:2369] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:39:34.476109 20273 tablet.cc:2369] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:39:34.616684 20299 raft_consensus.cc:482] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:34.617084 20299 raft_consensus.cc:504] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.619590 20299 leader_election.cc:288] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ac9e30c86b4be39354d7de52e67431 (127.18.186.129:40499), eb80a9f490ae40fc90079c986574ee78 (127.18.186.131:46253)
I20260808 23:39:34.627853 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.620153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:36251 (local address 127.18.186.131:46253)
0808 23:39:34.620454 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.620463 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.621391 (+ 928us) server_negotiation.cc:407] Connection header received
0808 23:39:34.621590 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.621598 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.621701 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.621819 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:34.622603 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.623942 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.626006 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.626830 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.626939 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.626990 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.627473 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.627599 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 23:39:34.628012 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.620148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:46253 (local address 127.18.186.130:36251)
0808 23:39:34.621270 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:34.621289 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:34.621444 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:34.621863 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:34.621871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:34.621911 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:34.622399 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.622413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.624063 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.624066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.625643 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.625655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.627150 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.627154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.627275 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.627300 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:39:34.627410 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 23:39:34.627561 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":2,"thread_start_us":815,"threads_started":1}
I20260808 23:39:34.628197 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.620011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:60503 (local address 127.18.186.129:40499)
0808 23:39:34.620453 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.620464 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.621451 (+ 987us) server_negotiation.cc:407] Connection header received
0808 23:39:34.621719 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.621725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.621833 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.621921 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:34.622903 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.624079 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.626173 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.627055 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.627251 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.627337 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.627759 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.627909 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260808 23:39:34.628376 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.620005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:40499 (local address 127.18.186.130:60503)
0808 23:39:34.621230 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:34.621256 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:34.621490 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:34.622072 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:34.622077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:34.622110 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:34.622693 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.622708 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.624239 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.624248 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.626014 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:34.626028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.627310 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.627317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:34.627574 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.627600 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 23:39:34.627730 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 23:39:34.627872 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":31,"thread_start_us":742,"threads_started":1}
I20260808 23:39:34.629396 19947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92424139fd2041b39d2081c532bdf644" candidate_uuid: "ebbc22a83f914374bda0102cc918b91d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ac9e30c86b4be39354d7de52e67431" is_pre_election: true
I20260808 23:39:34.630040 19947 raft_consensus.cc:2454] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebbc22a83f914374bda0102cc918b91d in term 0.
I20260808 23:39:34.630923 20020 leader_election.cc:302] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ac9e30c86b4be39354d7de52e67431, ebbc22a83f914374bda0102cc918b91d; no voters:
I20260808 23:39:34.631312 20299 raft_consensus.cc:2781] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:34.631466 20299 raft_consensus.cc:482] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:34.631567 20299 raft_consensus.cc:3037] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:34.632097 20225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92424139fd2041b39d2081c532bdf644" candidate_uuid: "ebbc22a83f914374bda0102cc918b91d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb80a9f490ae40fc90079c986574ee78" is_pre_election: true
I20260808 23:39:34.632638 20225 raft_consensus.cc:2454] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebbc22a83f914374bda0102cc918b91d in term 0.
I20260808 23:39:34.634465 20299 raft_consensus.cc:504] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.635059 20299 leader_election.cc:288] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [CANDIDATE]: Term 1 election: Requested vote from peers 53ac9e30c86b4be39354d7de52e67431 (127.18.186.129:40499), eb80a9f490ae40fc90079c986574ee78 (127.18.186.131:46253)
I20260808 23:39:34.635499 19947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92424139fd2041b39d2081c532bdf644" candidate_uuid: "ebbc22a83f914374bda0102cc918b91d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ac9e30c86b4be39354d7de52e67431"
I20260808 23:39:34.635638 20225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92424139fd2041b39d2081c532bdf644" candidate_uuid: "ebbc22a83f914374bda0102cc918b91d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb80a9f490ae40fc90079c986574ee78"
I20260808 23:39:34.635691 19947 raft_consensus.cc:3037] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:34.635888 20225 raft_consensus.cc:3037] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:34.638080 19947 raft_consensus.cc:2454] T 92424139fd2041b39d2081c532bdf644 P 53ac9e30c86b4be39354d7de52e67431 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebbc22a83f914374bda0102cc918b91d in term 1.
I20260808 23:39:34.638345 20225 raft_consensus.cc:2454] T 92424139fd2041b39d2081c532bdf644 P eb80a9f490ae40fc90079c986574ee78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebbc22a83f914374bda0102cc918b91d in term 1.
I20260808 23:39:34.638455 20020 leader_election.cc:302] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ac9e30c86b4be39354d7de52e67431, ebbc22a83f914374bda0102cc918b91d; no voters:
I20260808 23:39:34.638783 20299 raft_consensus.cc:2781] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:34.639182 20299 raft_consensus.cc:686] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [term 1 LEADER]: Becoming Leader. State: Replica: ebbc22a83f914374bda0102cc918b91d, State: Running, Role: LEADER
I20260808 23:39:34.639704 20299 consensus_queue.cc:237] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } }
I20260808 23:39:34.645213 19787 catalog_manager.cc:5495] T 92424139fd2041b39d2081c532bdf644 P ebbc22a83f914374bda0102cc918b91d reported cstate change: term changed from 0 to 1, leader changed from <none> to ebbc22a83f914374bda0102cc918b91d (127.18.186.130). New cstate: current_term: 1 leader_uuid: "ebbc22a83f914374bda0102cc918b91d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebbc22a83f914374bda0102cc918b91d" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ac9e30c86b4be39354d7de52e67431" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb80a9f490ae40fc90079c986574ee78" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 46253 } health_report { overall_health: UNKNOWN } } }
Aug 08 23:39:34 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:39:34.711213 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.696327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.18.186.129:40499)
0808 23:39:34.696610 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.696618 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.696940 (+ 322us) server_negotiation.cc:407] Connection header received
0808 23:39:34.697077 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.697080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.697146 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.697242 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:34.698834 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.699657 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.700988 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.701403 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.701495 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.705628 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:34.705643 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:34.705659 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:34.705693 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:34.708650 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.709263 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.709270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.709272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.709418 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.709756 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.709760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.709762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.710121 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:34.710358 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.710656 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.710819 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
Aug 08 23:39:34 dist-test-slave-5k54 krb5kdc[19738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232374, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:39:34 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232374, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:39:34.765057 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.749984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49612 (local address 127.18.186.129:40499)
0808 23:39:34.750194 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.750203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.750557 (+ 354us) server_negotiation.cc:407] Connection header received
0808 23:39:34.750777 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.750782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.750846 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.750958 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:34.752454 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.753188 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.754494 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.755131 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.755276 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.759424 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:34.759456 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:34.759463 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:34.759509 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:34.762369 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.762993 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.762997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.763000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.763135 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.763641 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.763644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.763647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.763919 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:34.764111 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.764408 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.764668 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 23:39:34 dist-test-slave-5k54 krb5kdc[19738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:39:34 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:39:34.812182 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.798472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49616 (local address 127.18.186.129:40499)
0808 23:39:34.798710 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.798717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.798997 (+ 280us) server_negotiation.cc:407] Connection header received
0808 23:39:34.799214 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.799219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.799284 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.799392 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:34.800813 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.801508 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.802748 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.803207 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.803348 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.807158 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:34.807163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:34.807166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:34.807198 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:34.809675 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.810328 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.810332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.810334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.810459 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.810849 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.810852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.810854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.811151 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:34.811334 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.811734 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.811930 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 23:39:34.814358 19902 tablet_service.cc:2908] Rejecting scan request for tablet 92424139fd2041b39d2081c532bdf644: Uninitialized: clean time has not yet been initialized
I20260808 23:39:34.818018 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 19857
I20260808 23:39:34.831529 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.819082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.18.186.129:40499)
0808 23:39:34.819338 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.819348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.819715 (+ 367us) server_negotiation.cc:407] Connection header received
0808 23:39:34.819858 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.819863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.819941 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.820030 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:34.821191 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.821952 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.823129 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.823676 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.823795 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.826094 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:34.826100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:34.826103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:34.826170 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:34.829029 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.829662 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.829666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.829668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.829782 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.830146 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.830149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.830151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.830426 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:34.830631 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.830867 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.831107 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:39:34.832235 19990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:34.928444 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 19857
I20260808 23:39:34.947512 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20001
Aug 08 23:39:34 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:39:34.963018 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:34.948356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.18.186.130:38177)
0808 23:39:34.948826 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:34.948831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:34.948848 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:39:34.948907 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:34.948912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:34.948971 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:34.949079 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:34.950383 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.951205 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.952632 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:34.953014 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:34.953106 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:34.957316 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:34.957331 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:34.957347 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:34.957386 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:34.960600 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.961245 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.961252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.961254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.961409 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:34.961743 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:34.961746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:34.961748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:34.962077 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:34.962302 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:34.962571 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 23:39:34.962739 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":248,"threads_started":1}
I20260808 23:39:34.963717 20128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:35.054981 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 20001
I20260808 23:39:35.074162 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20139
Aug 08 23:39:35 dist-test-slave-5k54 krb5kdc[19738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:39:35.089923 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:35.074981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41240 (local address 127.18.186.131:46253)
0808 23:39:35.075142 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:35.075147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:35.075164 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:39:35.075327 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:35.075331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:35.075406 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:35.075562 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:35.076901 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:35.077627 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:35.078960 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:35.079371 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:35.079881 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:35.084240 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:35.084256 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:35.084271 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:35.084306 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:35.087039 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:35.087834 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:35.087841 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:35.087843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:35.087975 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:35.088345 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:35.088351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:35.088354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:35.088730 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:35.089036 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:35.089340 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 23:39:35.089571 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:39:35.090591 20267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:35.178762 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 20139
I20260808 23:39:35.197736 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 19751
I20260808 23:39:35.198889 19824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:35.307076 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 19751
2026-08-08T23:39:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4445 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:39:35 dist-test-slave-5k54 krb5kdc[20323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:39:35 dist-test-slave-5k54 krb5kdc[20323](info): set up 2 sockets
Aug 08 23:39:35 dist-test-slave-5k54 krb5kdc[20323](info): commencing operation
krb5kdc: starting...
W20260808 23:39:36.450371 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.005s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:39:36 dist-test-slave-5k54 krb5kdc[20323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:39:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:39:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:39:36.654472 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:43343
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:43343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:36.977480 20336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:36.977881 20336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:36.978046 20336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:36.986061 20336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:39:36.986204 20336 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 23:39:36.986277 20336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:36.986368 20336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:36.986461 20336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:39:36.986548 20336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:39:36.996541 20336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:43343
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:43343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:36.997964 20336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:36.999317 20336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:37.011401 20341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:37.011492 20342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:37.011451 20344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:37.012681 20336 server_base.cc:1034] running on GCE node
I20260808 23:39:37.013516 20336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:37.014899 20336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:37.016150 20336 hybrid_clock.cc:648] HybridClock initialized: now 1786232377016089 us; error 41 us; skew 500 ppm
I20260808 23:39:37.016489 20336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:37 dist-test-slave-5k54 krb5kdc[20323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:37.019613 20336 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:39:37.020989 20336 webserver.cc:460] Webserver started at http://127.18.186.190:37945/ using document root <none> and password file <none>
I20260808 23:39:37.021701 20336 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:37.021834 20336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:37.022154 20336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:37.025142 20336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "41c9658cd06d4015a96910a61e2cf0ab"
format_stamp: "Formatted at 2026-08-08 23:39:37 on dist-test-slave-5k54"
server_key: "6a392ddd1d3b4f3d910493e4614d2b6e"
server_key_iv: "3cffc24380dd11ce671ce4f6a30446dc"
server_key_version: "encryptionkey@0"
I20260808 23:39:37.025913 20336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "41c9658cd06d4015a96910a61e2cf0ab"
format_stamp: "Formatted at 2026-08-08 23:39:37 on dist-test-slave-5k54"
server_key: "6a392ddd1d3b4f3d910493e4614d2b6e"
server_key_iv: "3cffc24380dd11ce671ce4f6a30446dc"
server_key_version: "encryptionkey@0"
I20260808 23:39:37.031073 20336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 23:39:37.034801 20351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:37.036163 20336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:39:37.036339 20336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "41c9658cd06d4015a96910a61e2cf0ab"
format_stamp: "Formatted at 2026-08-08 23:39:37 on dist-test-slave-5k54"
server_key: "6a392ddd1d3b4f3d910493e4614d2b6e"
server_key_iv: "3cffc24380dd11ce671ce4f6a30446dc"
server_key_version: "encryptionkey@0"
I20260808 23:39:37.036566 20336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:37.067267 20336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:37.071508 20336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:37.071781 20336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:37.093923 20336 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:43343
I20260808 23:39:37.093943 20415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:43343 every 8 connection(s)
I20260808 23:39:37.095284 20336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:39:37.098984 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:37.102970 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 20336
I20260808 23:39:37.103233 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:39:37.104135 19178 external_mini_cluster.cc:1428] Setting key 401307f737116517bb2eb9ce4b670144
I20260808 23:39:37.111003 20416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab: Bootstrap starting.
I20260808 23:39:37.114801 20416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:37.116081 20416 log.cc:824] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:37.119743 20416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab: No bootstrap required, opened a new log
Aug 08 23:39:37 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:37.130122 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:37.106301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.18.186.190:43343)
0808 23:39:37.107075 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:37.107086 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:37.107126 (+ 40us) server_negotiation.cc:407] Connection header received
0808 23:39:37.109433 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:37.109457 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:37.109844 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:37.110195 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:37.112091 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.113448 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.115252 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.116044 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.116194 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:37.121107 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:37.121121 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:37.121143 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:37.121194 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:37.125348 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:37.126051 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:37.126062 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:37.126076 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:37.126306 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:37.126690 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:37.126696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:37.126699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:37.127498 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:37.127796 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:37.128155 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 23:39:37.128649 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":362,"threads_started":1}
I20260808 23:39:37.130887 20416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c9658cd06d4015a96910a61e2cf0ab" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43343 } }
I20260808 23:39:37.131352 20416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:37.131524 20416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c9658cd06d4015a96910a61e2cf0ab, State: Initialized, Role: FOLLOWER
I20260808 23:39:37.132112 20416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c9658cd06d4015a96910a61e2cf0ab" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43343 } }
I20260808 23:39:37.132375 20416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:39:37.132534 20416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:39:37.132723 20416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:37.134905 20416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c9658cd06d4015a96910a61e2cf0ab" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43343 } }
I20260808 23:39:37.135473 20416 leader_election.cc:302] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41c9658cd06d4015a96910a61e2cf0ab; no voters:
I20260808 23:39:37.136169 20416 leader_election.cc:288] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:39:37.136289 20421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:37.137081 20421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [term 1 LEADER]: Becoming Leader. State: Replica: 41c9658cd06d4015a96910a61e2cf0ab, State: Running, Role: LEADER
I20260808 23:39:37.137624 20421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c9658cd06d4015a96910a61e2cf0ab" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43343 } }
I20260808 23:39:37.138430 20416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:39:37.140414 20422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41c9658cd06d4015a96910a61e2cf0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c9658cd06d4015a96910a61e2cf0ab" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43343 } } }
I20260808 23:39:37.140797 20422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:37.141664 20423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41c9658cd06d4015a96910a61e2cf0ab. Latest consensus state: current_term: 1 leader_uuid: "41c9658cd06d4015a96910a61e2cf0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c9658cd06d4015a96910a61e2cf0ab" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43343 } } }
I20260808 23:39:37.142087 20423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:37.144074 20426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:39:37.149329 20426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:39:37.158990 20426 catalog_manager.cc:1347] Generated new cluster ID: 74fcc3895d894a6cb23401e790d48d49
I20260808 23:39:37.159124 20426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:39:37.174197 20426 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:39:37.175588 20426 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:39:37.185096 20426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab: Generated new TSK 0
I20260808 23:39:37.185923 20426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:39:37.246580 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:43343
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:37.583662 20442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:37.583959 20442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:37.584057 20442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:37.591320 20442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:37.591468 20442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:37.591579 20442 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:39:37.591667 20442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:39:37.601501 20442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:37.602718 20442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:37.604022 20442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:37.617151 20448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:37.617249 20447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:37.617219 20450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:37.618263 20442 server_base.cc:1034] running on GCE node
I20260808 23:39:37.618793 20442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:37.619688 20442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:37.620944 20442 hybrid_clock.cc:648] HybridClock initialized: now 1786232377620869 us; error 62 us; skew 500 ppm
I20260808 23:39:37.621192 20442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:37 dist-test-slave-5k54 krb5kdc[20323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:37.629249 20442 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:39:37.630736 20442 webserver.cc:460] Webserver started at http://127.18.186.129:40439/ using document root <none> and password file <none>
I20260808 23:39:37.631556 20442 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:37.631672 20442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:37.632025 20442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:37.634981 20442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "119d15f2f32140e98db53151fd4ecae0"
format_stamp: "Formatted at 2026-08-08 23:39:37 on dist-test-slave-5k54"
server_key: "df0ed57a5afeb6649e406336f4c58d27"
server_key_iv: "d56d1291a489bb38f184d2258d789c0b"
server_key_version: "encryptionkey@0"
I20260808 23:39:37.635838 20442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "119d15f2f32140e98db53151fd4ecae0"
format_stamp: "Formatted at 2026-08-08 23:39:37 on dist-test-slave-5k54"
server_key: "df0ed57a5afeb6649e406336f4c58d27"
server_key_iv: "d56d1291a489bb38f184d2258d789c0b"
server_key_version: "encryptionkey@0"
I20260808 23:39:37.641485 20442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:39:37.645347 20457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:37.646864 20442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:39:37.647003 20442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "119d15f2f32140e98db53151fd4ecae0"
format_stamp: "Formatted at 2026-08-08 23:39:37 on dist-test-slave-5k54"
server_key: "df0ed57a5afeb6649e406336f4c58d27"
server_key_iv: "d56d1291a489bb38f184d2258d789c0b"
server_key_version: "encryptionkey@0"
I20260808 23:39:37.647253 20442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:37.662205 20442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:37.666662 20442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:37.666903 20442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:37.669826 20442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:37.669931 20442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:37.670096 20442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:37.670240 20442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 08 23:39:37 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:37.693284 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:37.670263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:37187 (local address 127.18.186.190:43343)
0808 23:39:37.670768 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:37.670777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:37.671982 (+ 1205us) server_negotiation.cc:407] Connection header received
0808 23:39:37.674068 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:37.674075 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:37.674165 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:37.674318 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:37.676574 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.677754 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.681151 (+ 3397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.681608 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.681758 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:37.686542 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:37.686547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:37.686550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:37.686585 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:37.689794 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:37.690678 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:37.690683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:37.690686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:37.690809 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:37.691267 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:37.691279 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:37.691281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:37.691622 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:37.691781 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:37.692685 (+ 904us) server_negotiation.cc:299] Negotiation successful
0808 23:39:37.692916 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":270,"threads_started":1}
I20260808 23:39:37.694893 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:37.670972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43343 (local address 127.18.186.129:37187)
0808 23:39:37.671828 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:37.671864 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:37.673729 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:37.674504 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:37.674512 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:37.675379 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:37.676375 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:37.676394 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.677959 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.677964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:37.680949 (+ 2985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:37.680974 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.681858 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.681868 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:37.682119 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:37.683098 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:37.683123 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:37.686255 (+ 3132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:37.690068 (+ 3813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:37.690077 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:37.690093 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:37.690533 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:37.690936 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:37.690941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:37.690945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:37.691143 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:37.691762 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:37.691769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:37.692306 (+ 537us) client_negotiation.cc:769] Sending connection context
0808 23:39:37.692592 (+ 286us) client_negotiation.cc:240] Negotiation successful
0808 23:39:37.693185 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":401,"threads_started":1}
I20260808 23:39:37.717281 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:37.701238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:35807 (local address 127.18.186.190:43343)
0808 23:39:37.701484 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:37.701493 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:37.702071 (+ 578us) server_negotiation.cc:407] Connection header received
0808 23:39:37.702246 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:37.702254 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:37.702370 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:37.702510 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:37.703477 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.704733 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.706097 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.707004 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.707167 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:37.710050 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:37.710055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:37.710058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:37.710088 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:37.713569 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:37.714697 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:37.714706 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:37.714711 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:37.714889 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:37.715386 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:37.715394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:37.715399 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:37.715769 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:37.715887 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:37.716558 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 23:39:37.716847 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:39:37.717837 20536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:37.701309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43343 (local address 127.18.186.129:35807)
0808 23:39:37.701974 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:37.701990 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:37.702100 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:37.702517 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:37.702520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:37.703026 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:37.703309 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:37.703317 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.704940 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.704943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:37.705956 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:37.705967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:37.707219 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:37.707224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:37.707352 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:37.708217 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:37.708245 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:37.709857 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:37.713707 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:37.713714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:37.713720 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:37.714555 (+ 835us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:37.715045 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:37.715053 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:37.715060 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:37.715256 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:37.715901 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:37.715909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:37.716193 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 23:39:37.716417 (+ 224us) client_negotiation.cc:240] Negotiation successful
0808 23:39:37.716908 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":356,"threads_started":1}
I20260808 23:39:37.722359 20442 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:36789
I20260808 23:39:37.722451 20587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:36789 every 8 connection(s)
I20260808 23:39:37.723822 20442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:39:37.725858 20588 heartbeater.cc:347] Connected to a master server at 127.18.186.190:43343
I20260808 23:39:37.726101 20588 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:37.726727 20588 heartbeater.cc:511] Master 127.18.186.190:43343 requested a full tablet report, sending...
I20260808 23:39:37.728634 20368 ts_manager.cc:194] Registered new tserver with Master: 119d15f2f32140e98db53151fd4ecae0 (127.18.186.129:36789)
I20260808 23:39:37.728845 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 20442
I20260808 23:39:37.729064 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:39:37.729696 19178 external_mini_cluster.cc:1428] Setting key f524ff5070d49c4eb46a491cdeefa70d
I20260808 23:39:37.730248 20368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:37187
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:39:37.772943 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:43343
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:38.109294 20592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:38.109638 20592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:38.109810 20592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:38.120018 20592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:38.120132 20592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:38.120218 20592 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:39:38.120304 20592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:39:38.130141 20592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:38.131362 20592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:38.132779 20592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:38.146608 20602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:38.146729 20600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:38.146631 20599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:38.152349 20592 server_base.cc:1034] running on GCE node
I20260808 23:39:38.153065 20592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:38.154031 20592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:38.155256 20592 hybrid_clock.cc:648] HybridClock initialized: now 1786232378155179 us; error 53 us; skew 500 ppm
I20260808 23:39:38.155607 20592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:38 dist-test-slave-5k54 krb5kdc[20323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:38.158766 20592 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:39:38.160307 20592 webserver.cc:460] Webserver started at http://127.18.186.130:41223/ using document root <none> and password file <none>
I20260808 23:39:38.161038 20592 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:38.161170 20592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:38.161507 20592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:38.164476 20592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "619f0b12894c46eab2c1cf90aeeae343"
format_stamp: "Formatted at 2026-08-08 23:39:38 on dist-test-slave-5k54"
server_key: "ae755e9d4b1e94072c8f790134de5a07"
server_key_iv: "fcf50da5d79d0f319f180e80e7d303ec"
server_key_version: "encryptionkey@0"
I20260808 23:39:38.165283 20592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "619f0b12894c46eab2c1cf90aeeae343"
format_stamp: "Formatted at 2026-08-08 23:39:38 on dist-test-slave-5k54"
server_key: "ae755e9d4b1e94072c8f790134de5a07"
server_key_iv: "fcf50da5d79d0f319f180e80e7d303ec"
server_key_version: "encryptionkey@0"
I20260808 23:39:38.170467 20592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:39:38.173916 20610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:38.175194 20592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:39:38.175359 20592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "619f0b12894c46eab2c1cf90aeeae343"
format_stamp: "Formatted at 2026-08-08 23:39:38 on dist-test-slave-5k54"
server_key: "ae755e9d4b1e94072c8f790134de5a07"
server_key_iv: "fcf50da5d79d0f319f180e80e7d303ec"
server_key_version: "encryptionkey@0"
I20260808 23:39:38.175648 20592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:38.198066 20592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:38.202143 20592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:38.202445 20592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:38.205519 20592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:38.205633 20592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:38.205754 20592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:38.205829 20592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:39:38 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:38.230258 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.206238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:54395 (local address 127.18.186.190:43343)
0808 23:39:38.206451 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:38.206457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:38.208202 (+ 1745us) server_negotiation.cc:407] Connection header received
0808 23:39:38.210290 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:38.210297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:38.210393 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:38.210507 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:38.212930 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.214066 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.216737 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.217197 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.217336 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.222166 (+ 4830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:38.222172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:38.222175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:38.222211 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:38.225924 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.226951 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.226957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.226963 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.227093 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.227745 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.227748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.227750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.228149 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:38.228301 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:38.229398 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0808 23:39:38.229720 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:39:38.230988 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.206987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43343 (local address 127.18.186.130:54395)
0808 23:39:38.208020 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:38.208085 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:38.209998 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:38.210801 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:38.210818 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:38.211702 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:38.212716 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.212751 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.214236 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.214247 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.216584 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.216596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.217449 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.217457 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.217687 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.218477 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:38.218520 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:38.221929 (+ 3409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:38.226123 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:38.226133 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:38.226153 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:38.226820 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:38.227212 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:38.227217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:38.227220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:38.227606 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:38.228362 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:38.228368 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:38.228919 (+ 551us) client_negotiation.cc:769] Sending connection context
0808 23:39:38.229247 (+ 328us) client_negotiation.cc:240] Negotiation successful
0808 23:39:38.229730 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":446,"threads_started":1}
I20260808 23:39:38.252430 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.237160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:47347 (local address 127.18.186.190:43343)
0808 23:39:38.237389 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:38.237398 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:38.238029 (+ 631us) server_negotiation.cc:407] Connection header received
0808 23:39:38.238226 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:38.238230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:38.238298 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:38.238377 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:38.239911 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.240756 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.243184 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.243658 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.243789 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.246231 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:38.246237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:38.246240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:38.246269 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:38.249286 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.250217 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.250224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.250231 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.250377 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.250838 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.250845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.250850 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.251232 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:38.251388 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:38.251866 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 23:39:38.252093 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:39:38.252573 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.237133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43343 (local address 127.18.186.130:47347)
0808 23:39:38.237899 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:38.237925 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:38.238104 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:38.238444 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:38.238452 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:38.239214 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:38.239748 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.239765 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.240911 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.240920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.243016 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.243035 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.243870 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.243876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.244063 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.244796 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:38.244825 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:38.246027 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:38.249446 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:38.249452 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:38.249455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:38.250081 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:38.250498 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:38.250503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:38.250506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:38.250674 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:38.251353 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:38.251358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:38.251589 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 23:39:38.251769 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 23:39:38.252125 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":388,"threads_started":1}
I20260808 23:39:38.265331 20592 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:40327
I20260808 23:39:38.265359 20737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:40327 every 8 connection(s)
I20260808 23:39:38.266592 20592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:39:38.268759 20739 heartbeater.cc:347] Connected to a master server at 127.18.186.190:43343
I20260808 23:39:38.268953 20739 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:38.269520 20739 heartbeater.cc:511] Master 127.18.186.190:43343 requested a full tablet report, sending...
I20260808 23:39:38.270766 20367 ts_manager.cc:194] Registered new tserver with Master: 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327)
I20260808 23:39:38.271552 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 20592
I20260808 23:39:38.271767 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:39:38.271776 20367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:54395
I20260808 23:39:38.272243 19178 external_mini_cluster.cc:1428] Setting key 845f74b76134be2d06a5532b1ef4702d
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:39:38.316026 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:43343
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:38.651321 20743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:38.651762 20743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:38.651952 20743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:38.659353 20743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:38.659541 20743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:38.659698 20743 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:39:38.659844 20743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:39:38.669677 20743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:38.671001 20743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:38.672360 20743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:38.685571 20749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:38.685657 20752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:38.685633 20750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:38.686501 20743 server_base.cc:1034] running on GCE node
I20260808 23:39:38.687124 20743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:38.688200 20743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:38.689442 20743 hybrid_clock.cc:648] HybridClock initialized: now 1786232378689383 us; error 39 us; skew 500 ppm
I20260808 23:39:38.689754 20743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:38 dist-test-slave-5k54 krb5kdc[20323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:38.693092 20743 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:39:38.694514 20743 webserver.cc:460] Webserver started at http://127.18.186.131:45271/ using document root <none> and password file <none>
I20260808 23:39:38.695215 20743 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:38.695350 20743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:38.695700 20743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:38.698678 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "8ce3da58981442c89ab406c79c737de0"
format_stamp: "Formatted at 2026-08-08 23:39:38 on dist-test-slave-5k54"
server_key: "c50c518d5c0e4a74113aa4bc8243524c"
server_key_iv: "d0be54e444c816aa634a55bd45c7648c"
server_key_version: "encryptionkey@0"
I20260808 23:39:38.699538 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "8ce3da58981442c89ab406c79c737de0"
format_stamp: "Formatted at 2026-08-08 23:39:38 on dist-test-slave-5k54"
server_key: "c50c518d5c0e4a74113aa4bc8243524c"
server_key_iv: "d0be54e444c816aa634a55bd45c7648c"
server_key_version: "encryptionkey@0"
I20260808 23:39:38.704826 20743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 23:39:38.708240 20759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:38.709539 20743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:39:38.709720 20743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "8ce3da58981442c89ab406c79c737de0"
format_stamp: "Formatted at 2026-08-08 23:39:38 on dist-test-slave-5k54"
server_key: "c50c518d5c0e4a74113aa4bc8243524c"
server_key_iv: "d0be54e444c816aa634a55bd45c7648c"
server_key_version: "encryptionkey@0"
I20260808 23:39:38.709944 20743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:38.725003 20743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:38.729319 20743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:38.729598 20743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:38.732381 20743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:38.732496 20743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:38.732686 20743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:38.732821 20743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 23:39:38.733266 20588 heartbeater.cc:503] Master 127.18.186.190:43343 was elected leader, sending a full tablet report...
Aug 08 23:39:38 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:38.757858 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.732919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:38301 (local address 127.18.186.190:43343)
0808 23:39:38.733228 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:38.733238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:38.734819 (+ 1581us) server_negotiation.cc:407] Connection header received
0808 23:39:38.737741 (+ 2922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:38.737747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:38.737843 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:38.737903 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:38.740521 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.741737 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.743544 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.744335 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.744522 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.749966 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:38.749973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:38.749978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:38.750022 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:38.753798 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.754896 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.754904 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.754910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.755072 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.755677 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.755682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.755685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.755960 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:38.756140 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:38.757243 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0808 23:39:38.757484 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260808 23:39:38.759541 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.733610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43343 (local address 127.18.186.131:38301)
0808 23:39:38.734656 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:38.734701 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:38.737402 (+ 2701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:38.738473 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:38.738481 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:38.739270 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:38.740249 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.740285 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.741903 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.741908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.743313 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.743326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.744924 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.744935 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.745282 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.746231 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:38.746271 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:38.749704 (+ 3433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:38.753958 (+ 4254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:38.753967 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:38.753986 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:38.754697 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:38.755198 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:38.755202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:38.755207 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:38.755549 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:38.756266 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:38.756275 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:38.756747 (+ 472us) client_negotiation.cc:769] Sending connection context
0808 23:39:38.757124 (+ 377us) client_negotiation.cc:240] Negotiation successful
0808 23:39:38.757706 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"spinlock_wait_cycles":26368,"thread_start_us":654,"threads_started":1}
I20260808 23:39:38.781463 20743 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:40109
I20260808 23:39:38.781490 20885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:40109 every 8 connection(s)
I20260808 23:39:38.782065 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.766223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:51187 (local address 127.18.186.190:43343)
0808 23:39:38.766449 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:38.766458 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:38.767073 (+ 615us) server_negotiation.cc:407] Connection header received
0808 23:39:38.767263 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:38.767267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:38.767323 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:38.767418 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:38.768981 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.769679 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.772327 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.772745 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.772840 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.775460 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:38.775467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:38.775472 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:38.775525 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:38.779014 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.779927 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.779933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.779937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.780069 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.780591 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.780596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.780599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.780880 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:38.781035 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:38.781483 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 23:39:38.781709 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:39:38.782192 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.766219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43343 (local address 127.18.186.131:51187)
0808 23:39:38.766949 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:38.766975 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:38.767142 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:38.767495 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:38.767503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:38.768244 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:38.768781 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.768799 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.769862 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.769870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.772193 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.772207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.772929 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.772939 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.773184 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.774036 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:38.774071 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:38.775235 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:38.779231 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:38.779239 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:38.779243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:38.779783 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:38.780216 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:38.780221 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:38.780228 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:38.780385 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:38.781009 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:38.781014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:38.781207 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 23:39:38.781365 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 23:39:38.781709 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":373,"threads_started":1}
I20260808 23:39:38.783085 20743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:39:38.785254 20886 heartbeater.cc:347] Connected to a master server at 127.18.186.190:43343
I20260808 23:39:38.785588 20886 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:38.786119 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 20743
I20260808 23:39:38.786319 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:39:38.786409 20886 heartbeater.cc:511] Master 127.18.186.190:43343 requested a full tablet report, sending...
I20260808 23:39:38.786793 19178 external_mini_cluster.cc:1428] Setting key ef267ba77624605e3b108e96a8697866
I20260808 23:39:38.787845 20367 ts_manager.cc:194] Registered new tserver with Master: 8ce3da58981442c89ab406c79c737de0 (127.18.186.131:40109)
I20260808 23:39:38.788637 20367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:38301
I20260808 23:39:38.790686 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:39:38 dist-test-slave-5k54 krb5kdc[20323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:39:38 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:38.840250 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.825805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.18.186.190:43343)
0808 23:39:38.826059 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:38.826066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:38.826194 (+ 128us) server_negotiation.cc:407] Connection header received
0808 23:39:38.826350 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:38.826355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:38.826425 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:38.826519 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:38.827976 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.828707 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.829971 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.830371 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.830464 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.834878 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:38.834884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:38.834887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:38.834928 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:38.837516 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.838280 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.838283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.838286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.838485 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:38.838815 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:38.838818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:38.838820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:38.839150 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:38.839339 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:38.839672 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 23:39:38.839843 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 23:39:38.844534 20367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:39:38.851881 20367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:38.901119 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.864384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.18.186.130:40327)
0808 23:39:38.892287 (+ 27903us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:38.892294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:38.892318 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:39:38.892400 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:38.892406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:38.892613 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:38.892837 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:38.894218 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.895777 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.898611 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.900019 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.900196 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.900408 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:38.900664 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 23:39:38.900831 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27753,"thread_start_us":27649,"threads_started":1}
I20260808 23:39:38.901167 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.864253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:40327 (local address 127.0.0.1:53614)
0808 23:39:38.890007 (+ 25754us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:38.890077 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:38.890312 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:38.892835 (+ 2523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:38.892843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:38.892872 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:38.893605 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.893621 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.895941 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.895947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.898436 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.898451 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.900214 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.900220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.900416 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.900479 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 23:39:38.900610 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 23:39:38.900801 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25369,"thread_start_us":29709,"threads_started":1}
I20260808 23:39:38.901971 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.864358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36789 (local address 127.0.0.1:58532)
0808 23:39:38.893651 (+ 29293us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:38.893676 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:38.893894 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:38.894798 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:38.894803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:38.894827 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:38.895194 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.895205 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.897083 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.897088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.899152 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.899163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.901116 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.901123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.901290 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.901321 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:39:38.901425 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 23:39:38.901569 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29138,"mutex_wait_us":107,"thread_start_us":28823,"threads_started":1}
I20260808 23:39:38.902597 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.864787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:40109 (local address 127.0.0.1:35138)
0808 23:39:38.894287 (+ 29500us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:38.894303 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:38.894446 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:38.895513 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:38.895520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:38.895552 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:38.896040 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.896055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.897668 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.897675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.899738 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:38.899752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.901846 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.901854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:38.902055 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.902091 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:39:38.902171 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 23:39:38.902298 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29372,"thread_start_us":26339,"threads_started":1}
I20260808 23:39:38.902788 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.865067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.18.186.129:36789)
0808 23:39:38.891082 (+ 26015us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:38.891095 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:38.893826 (+ 2731us) server_negotiation.cc:407] Connection header received
0808 23:39:38.894130 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:38.894141 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:38.894411 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:38.894592 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:38.895366 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.896920 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.899296 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.900620 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.900792 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.900967 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:38.902299 (+ 1332us) server_negotiation.cc:299] Negotiation successful
0808 23:39:38.902490 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25798,"thread_start_us":25623,"threads_started":1}
I20260808 23:39:38.903405 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:38.865341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.18.186.131:40109)
0808 23:39:38.889008 (+ 23667us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:38.889022 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:38.894424 (+ 5402us) server_negotiation.cc:407] Connection header received
0808 23:39:38.894589 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:38.894597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:38.894805 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:38.895012 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:38.896256 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.897518 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.899912 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:38.901350 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:38.901541 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:38.901695 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:38.902919 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0808 23:39:38.903097 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23389,"thread_start_us":23698,"threads_started":1}
I20260808 23:39:38.904992 20665 tablet_service.cc:1467] Processing CreateTablet for tablet f9aeacbf0b7a4faf8da033d33a15fbbf (DEFAULT_TABLE table=test-table [id=2eb2c55341e544f6912bcfd60064e51b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:38.906198 20665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9aeacbf0b7a4faf8da033d33a15fbbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:38.906440 20513 tablet_service.cc:1467] Processing CreateTablet for tablet f9aeacbf0b7a4faf8da033d33a15fbbf (DEFAULT_TABLE table=test-table [id=2eb2c55341e544f6912bcfd60064e51b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:38.906996 20813 tablet_service.cc:1467] Processing CreateTablet for tablet f9aeacbf0b7a4faf8da033d33a15fbbf (DEFAULT_TABLE table=test-table [id=2eb2c55341e544f6912bcfd60064e51b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:38.907846 20513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9aeacbf0b7a4faf8da033d33a15fbbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:38.908393 20813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9aeacbf0b7a4faf8da033d33a15fbbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:38.917927 20907 tablet_bootstrap.cc:492] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Bootstrap starting.
I20260808 23:39:38.921003 20907 tablet_bootstrap.cc:654] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:38.922197 20908 tablet_bootstrap.cc:492] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Bootstrap starting.
I20260808 23:39:38.923081 20907 log.cc:824] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:38.924299 20909 tablet_bootstrap.cc:492] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Bootstrap starting.
I20260808 23:39:38.926543 20908 tablet_bootstrap.cc:654] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:38.927625 20908 log.cc:824] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:38.927661 20907 tablet_bootstrap.cc:492] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: No bootstrap required, opened a new log
I20260808 23:39:38.927925 20909 tablet_bootstrap.cc:654] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:38.927954 20907 ts_tablet_manager.cc:1397] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260808 23:39:38.929131 20909 log.cc:824] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:38.932204 20909 tablet_bootstrap.cc:492] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: No bootstrap required, opened a new log
I20260808 23:39:38.932204 20908 tablet_bootstrap.cc:492] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: No bootstrap required, opened a new log
I20260808 23:39:38.932482 20908 ts_tablet_manager.cc:1397] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 23:39:38.932476 20909 ts_tablet_manager.cc:1397] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 23:39:38.936977 20907 raft_consensus.cc:348] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:38.937359 20907 raft_consensus.cc:374] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:38.937487 20907 raft_consensus.cc:729] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619f0b12894c46eab2c1cf90aeeae343, State: Initialized, Role: FOLLOWER
I20260808 23:39:38.938130 20907 consensus_queue.cc:260] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:38.939257 20739 heartbeater.cc:503] Master 127.18.186.190:43343 was elected leader, sending a full tablet report...
I20260808 23:39:38.940006 20907 ts_tablet_manager.cc:1428] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 23:39:38.943465 20909 raft_consensus.cc:348] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:38.943473 20908 raft_consensus.cc:348] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:38.943897 20908 raft_consensus.cc:374] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:38.943893 20909 raft_consensus.cc:374] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:38.944000 20909 raft_consensus.cc:729] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce3da58981442c89ab406c79c737de0, State: Initialized, Role: FOLLOWER
I20260808 23:39:38.944001 20908 raft_consensus.cc:729] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119d15f2f32140e98db53151fd4ecae0, State: Initialized, Role: FOLLOWER
I20260808 23:39:38.944591 20909 consensus_queue.cc:260] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:38.944592 20908 consensus_queue.cc:260] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:38.946249 20886 heartbeater.cc:503] Master 127.18.186.190:43343 was elected leader, sending a full tablet report...
I20260808 23:39:38.946833 20909 ts_tablet_manager.cc:1428] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260808 23:39:38.947762 20908 ts_tablet_manager.cc:1428] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260808 23:39:38.976485 20589 tablet.cc:2369] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:39:39.018664 20740 tablet.cc:2369] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:39:39.024241 20915 raft_consensus.cc:482] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:39.024626 20915 raft_consensus.cc:504] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.026675 20915 leader_election.cc:288] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327), 119d15f2f32140e98db53151fd4ecae0 (127.18.186.129:36789)
I20260808 23:39:39.030529 20913 raft_consensus.cc:482] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:39.030810 20913 raft_consensus.cc:504] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
W20260808 23:39:39.034883 20888 tablet.cc:2369] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:39:39.034998 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.027041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36789 (local address 127.18.186.131:42899)
0808 23:39:39.027319 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.027336 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.027510 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.027956 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.027970 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.028026 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:39.028237 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.028246 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.029574 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.029578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.031723 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.031738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.033978 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.033987 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.034290 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.034347 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.034461 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.034646 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"spinlock_wait_cycles":30336}
I20260808 23:39:39.035691 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.027364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:42899 (local address 127.18.186.129:36789)
0808 23:39:39.027623 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.027630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.027652 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:39:39.027725 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.027731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.027823 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.027907 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:39.028412 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.029437 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.031905 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.033270 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.033434 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.033509 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.035258 (+ 1749us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.035395 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 23:39:39.036747 20913 leader_election.cc:288] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 119d15f2f32140e98db53151fd4ecae0 (127.18.186.129:36789), 8ce3da58981442c89ab406c79c737de0 (127.18.186.131:40109)
I20260808 23:39:39.038196 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9aeacbf0b7a4faf8da033d33a15fbbf" candidate_uuid: "8ce3da58981442c89ab406c79c737de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119d15f2f32140e98db53151fd4ecae0" is_pre_election: true
I20260808 23:39:39.038864 20538 raft_consensus.cc:2454] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ce3da58981442c89ab406c79c737de0 in term 0.
I20260808 23:39:39.039645 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.027028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:52759 (local address 127.18.186.130:40327)
0808 23:39:39.027247 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.027255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.028002 (+ 747us) server_negotiation.cc:407] Connection header received
0808 23:39:39.028178 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.028181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.028238 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.028303 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:39.028820 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.029769 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.032109 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.033532 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.033740 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.033859 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.039143 (+ 5284us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.039336 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":71}
I20260808 23:39:39.040104 20761 leader_election.cc:302] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119d15f2f32140e98db53151fd4ecae0, 8ce3da58981442c89ab406c79c737de0; no voters:
I20260808 23:39:39.040587 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.027069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:40327 (local address 127.18.186.131:52759)
0808 23:39:39.027857 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.027894 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.028036 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.028354 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.028359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.028395 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:39.028662 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.028668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.029897 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.029902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.031938 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.031948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.038201 (+ 6253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.038209 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.038918 (+ 709us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.038984 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.039937 (+ 953us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.040082 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"spinlock_wait_cycles":10624,"thread_start_us":479,"threads_started":1}
I20260808 23:39:39.040679 20915 raft_consensus.cc:2781] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:39.040849 20915 raft_consensus.cc:482] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:39.040964 20915 raft_consensus.cc:3037] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.041613 20692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9aeacbf0b7a4faf8da033d33a15fbbf" candidate_uuid: "8ce3da58981442c89ab406c79c737de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619f0b12894c46eab2c1cf90aeeae343" is_pre_election: true
I20260808 23:39:39.042104 20692 raft_consensus.cc:2454] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ce3da58981442c89ab406c79c737de0 in term 0.
I20260808 23:39:39.044193 20915 raft_consensus.cc:504] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.044806 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.033731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:54805 (local address 127.18.186.129:36789)
0808 23:39:39.035944 (+ 2213us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.035949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.036923 (+ 974us) server_negotiation.cc:407] Connection header received
0808 23:39:39.037219 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.037226 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.037329 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.037416 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:39.038497 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.039721 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.042585 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.043853 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.044046 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.044113 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.044381 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.044509 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2095,"thread_start_us":3448,"threads_started":1}
I20260808 23:39:39.044971 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.032756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36789 (local address 127.18.186.130:54805)
0808 23:39:39.036797 (+ 4041us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.036827 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.037039 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.037652 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.037660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.037720 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:39.038316 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.038332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.040086 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.040092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.042376 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.042390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.044072 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.044078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.044222 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.044245 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.044347 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.044523 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3841,"thread_start_us":5137,"threads_started":1}
I20260808 23:39:39.045279 20692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9aeacbf0b7a4faf8da033d33a15fbbf" candidate_uuid: "8ce3da58981442c89ab406c79c737de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619f0b12894c46eab2c1cf90aeeae343"
I20260808 23:39:39.045521 20692 raft_consensus.cc:3037] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.045831 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9aeacbf0b7a4faf8da033d33a15fbbf" candidate_uuid: "619f0b12894c46eab2c1cf90aeeae343" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119d15f2f32140e98db53151fd4ecae0" is_pre_election: true
I20260808 23:39:39.045910 20915 leader_election.cc:288] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [CANDIDATE]: Term 1 election: Requested vote from peers 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327), 119d15f2f32140e98db53151fd4ecae0 (127.18.186.129:36789)
I20260808 23:39:39.046077 20538 raft_consensus.cc:2454] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 619f0b12894c46eab2c1cf90aeeae343 in term 0.
I20260808 23:39:39.046275 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.033479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:35637 (local address 127.18.186.131:40109)
0808 23:39:39.033671 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.033677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.038408 (+ 4731us) server_negotiation.cc:407] Connection header received
0808 23:39:39.038665 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.038670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.038735 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.038797 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:39.040007 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.041222 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.043863 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.045197 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.045371 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.045439 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.045840 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.046007 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:39:39.046721 20612 leader_election.cc:302] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119d15f2f32140e98db53151fd4ecae0, 619f0b12894c46eab2c1cf90aeeae343; no voters:
I20260808 23:39:39.047189 20538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9aeacbf0b7a4faf8da033d33a15fbbf" candidate_uuid: "8ce3da58981442c89ab406c79c737de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119d15f2f32140e98db53151fd4ecae0"
I20260808 23:39:39.047297 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.033032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:40109 (local address 127.18.186.130:35637)
0808 23:39:39.038307 (+ 5275us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.038327 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.038508 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.039008 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.039014 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.039043 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:39.039788 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.039804 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.041480 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.041486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.043693 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.043708 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.045438 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.045445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.045638 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.045682 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.046437 (+ 755us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.046568 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5066,"mutex_wait_us":110,"thread_start_us":3310,"threads_started":1}
I20260808 23:39:39.047407 20538 raft_consensus.cc:3037] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.048913 20692 raft_consensus.cc:2454] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ce3da58981442c89ab406c79c737de0 in term 1.
I20260808 23:39:39.049077 20913 raft_consensus.cc:2741] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260808 23:39:39.049782 20538 raft_consensus.cc:2454] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ce3da58981442c89ab406c79c737de0 in term 1.
I20260808 23:39:39.050239 20761 leader_election.cc:302] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119d15f2f32140e98db53151fd4ecae0, 8ce3da58981442c89ab406c79c737de0; no voters:
I20260808 23:39:39.050541 20915 raft_consensus.cc:2781] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:39.050768 20833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9aeacbf0b7a4faf8da033d33a15fbbf" candidate_uuid: "619f0b12894c46eab2c1cf90aeeae343" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce3da58981442c89ab406c79c737de0" is_pre_election: true
I20260808 23:39:39.051203 20915 raft_consensus.cc:686] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 1 LEADER]: Becoming Leader. State: Replica: 8ce3da58981442c89ab406c79c737de0, State: Running, Role: LEADER
I20260808 23:39:39.051847 20915 consensus_queue.cc:237] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.056432 20368 catalog_manager.cc:5495] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ce3da58981442c89ab406c79c737de0 (127.18.186.131). New cstate: current_term: 1 leader_uuid: "8ce3da58981442c89ab406c79c737de0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } health_report { overall_health: HEALTHY } } }
I20260808 23:39:39.086591 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.080436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.18.186.131:40109)
0808 23:39:39.080633 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.080640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.080738 (+ 98us) server_negotiation.cc:407] Connection header received
0808 23:39:39.081192 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.081197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.081303 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.081447 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:39.081825 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.082604 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.084209 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.084637 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.084814 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.085306 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:39.085901 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:39.086064 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.086203 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.086291 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:39:39.095925 20538 raft_consensus.cc:1261] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Refusing update from remote peer 8ce3da58981442c89ab406c79c737de0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:39.096989 20915 consensus_queue.cc:1035] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:39.098182 20692 raft_consensus.cc:1261] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Refusing update from remote peer 8ce3da58981442c89ab406c79c737de0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:39.099082 20915 consensus_queue.cc:1035] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:39.109267 20923 mvcc.cc:205] Tried to move back new op lower bound from 7316407824764301312 to 7316407824600297472. Current Snapshot: MvccSnapshot[applied={T|T < 7316407824764301312}]
I20260808 23:39:39.110605 20924 mvcc.cc:205] Tried to move back new op lower bound from 7316407824764301312 to 7316407824600297472. Current Snapshot: MvccSnapshot[applied={T|T < 7316407824764301312}]
I20260808 23:39:39.136039 20367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:39.136404 20367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:39.140769 20367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab: Sending DeleteTablet for 3 replicas of tablet f9aeacbf0b7a4faf8da033d33a15fbbf
I20260808 23:39:39.142179 20665 tablet_service.cc:1514] Processing DeleteTablet for tablet f9aeacbf0b7a4faf8da033d33a15fbbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:39 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53614
I20260808 23:39:39.142318 20513 tablet_service.cc:1514] Processing DeleteTablet for tablet f9aeacbf0b7a4faf8da033d33a15fbbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:39 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:58532
I20260808 23:39:39.143146 20813 tablet_service.cc:1514] Processing DeleteTablet for tablet f9aeacbf0b7a4faf8da033d33a15fbbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:39 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:35138
I20260808 23:39:39.144878 20367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:39:39.145789 20367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:39.154309 20812 tablet_service.cc:1467] Processing CreateTablet for tablet 2053abe6a8fe4503836aeb95a53cd39e (DEFAULT_TABLE table=test-table [id=a228fcb48f444c949bdcd499ad98423c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:39.155059 20812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2053abe6a8fe4503836aeb95a53cd39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:39.155273 20934 tablet_replica.cc:330] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: stopping tablet replica
I20260808 23:39:39.156049 20934 raft_consensus.cc:2229] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:39:39.156235 20664 tablet_service.cc:1467] Processing CreateTablet for tablet 2053abe6a8fe4503836aeb95a53cd39e (DEFAULT_TABLE table=test-table [id=a228fcb48f444c949bdcd499ad98423c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:39.156849 20664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2053abe6a8fe4503836aeb95a53cd39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:39.156966 20934 raft_consensus.cc:2258] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:39.159468 20932 tablet_replica.cc:330] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: stopping tablet replica
I20260808 23:39:39.160023 20932 raft_consensus.cc:2229] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:39.160560 20932 raft_consensus.cc:2258] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:39.160610 20512 tablet_service.cc:1467] Processing CreateTablet for tablet 2053abe6a8fe4503836aeb95a53cd39e (DEFAULT_TABLE table=test-table [id=a228fcb48f444c949bdcd499ad98423c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:39.161221 20512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2053abe6a8fe4503836aeb95a53cd39e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:39.165493 20934 ts_tablet_manager.cc:1905] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:39.167224 20908 tablet_bootstrap.cc:492] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: Bootstrap starting.
I20260808 23:39:39.168908 20908 tablet_bootstrap.cc:654] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:39.173055 20932 ts_tablet_manager.cc:1905] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:39.173816 20909 tablet_bootstrap.cc:492] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: Bootstrap starting.
I20260808 23:39:39.174243 20907 tablet_bootstrap.cc:492] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: Bootstrap starting.
I20260808 23:39:39.175918 20907 tablet_bootstrap.cc:654] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:39.175957 20909 tablet_bootstrap.cc:654] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:39.181136 20908 tablet_bootstrap.cc:492] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: No bootstrap required, opened a new log
I20260808 23:39:39.181296 20908 ts_tablet_manager.cc:1397] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.007s
I20260808 23:39:39.182261 20908 raft_consensus.cc:348] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.182552 20908 raft_consensus.cc:374] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:39.182659 20908 raft_consensus.cc:729] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119d15f2f32140e98db53151fd4ecae0, State: Initialized, Role: FOLLOWER
I20260808 23:39:39.183027 20908 consensus_queue.cc:260] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.184585 20908 ts_tablet_manager.cc:1428] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 23:39:39.187333 20933 tablet_replica.cc:330] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: stopping tablet replica
I20260808 23:39:39.187842 20933 raft_consensus.cc:2229] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:39.188319 20933 raft_consensus.cc:2258] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:39.189419 20909 tablet_bootstrap.cc:492] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: No bootstrap required, opened a new log
I20260808 23:39:39.189584 20909 ts_tablet_manager.cc:1397] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.007s
I20260808 23:39:39.190145 20907 tablet_bootstrap.cc:492] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: No bootstrap required, opened a new log
I20260808 23:39:39.190294 20907 ts_tablet_manager.cc:1397] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260808 23:39:39.190410 20909 raft_consensus.cc:348] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.190650 20909 raft_consensus.cc:374] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:39.190760 20909 raft_consensus.cc:729] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce3da58981442c89ab406c79c737de0, State: Initialized, Role: FOLLOWER
I20260808 23:39:39.191035 20909 consensus_queue.cc:260] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.191452 20907 raft_consensus.cc:348] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.191692 20907 raft_consensus.cc:374] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:39.191798 20907 raft_consensus.cc:729] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619f0b12894c46eab2c1cf90aeeae343, State: Initialized, Role: FOLLOWER
I20260808 23:39:39.191879 20909 ts_tablet_manager.cc:1428] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:39:39.192051 20907 consensus_queue.cc:260] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.194407 20934 ts_tablet_manager.cc:1918] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:39:39.194660 20934 log.cc:1196] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/f9aeacbf0b7a4faf8da033d33a15fbbf
I20260808 23:39:39.194715 20932 ts_tablet_manager.cc:1918] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:39:39.194914 20932 log.cc:1196] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/f9aeacbf0b7a4faf8da033d33a15fbbf
I20260808 23:39:39.195218 20934 ts_tablet_manager.cc:1939] T f9aeacbf0b7a4faf8da033d33a15fbbf P 8ce3da58981442c89ab406c79c737de0: Deleting consensus metadata
I20260808 23:39:39.195578 20907 ts_tablet_manager.cc:1428] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260808 23:39:39.195675 20932 ts_tablet_manager.cc:1939] T f9aeacbf0b7a4faf8da033d33a15fbbf P 119d15f2f32140e98db53151fd4ecae0: Deleting consensus metadata
I20260808 23:39:39.197162 20355 catalog_manager.cc:4841] TS 8ce3da58981442c89ab406c79c737de0 (127.18.186.131:40109): tablet f9aeacbf0b7a4faf8da033d33a15fbbf (table test-table [id=2eb2c55341e544f6912bcfd60064e51b]) successfully deleted
I20260808 23:39:39.197531 20353 catalog_manager.cc:4841] TS 119d15f2f32140e98db53151fd4ecae0 (127.18.186.129:36789): tablet f9aeacbf0b7a4faf8da033d33a15fbbf (table test-table [id=2eb2c55341e544f6912bcfd60064e51b]) successfully deleted
I20260808 23:39:39.198565 20933 ts_tablet_manager.cc:1905] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:39.203282 20933 ts_tablet_manager.cc:1918] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:39:39.203498 20933 log.cc:1196] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/f9aeacbf0b7a4faf8da033d33a15fbbf
I20260808 23:39:39.204011 20933 ts_tablet_manager.cc:1939] T f9aeacbf0b7a4faf8da033d33a15fbbf P 619f0b12894c46eab2c1cf90aeeae343: Deleting consensus metadata
I20260808 23:39:39.205703 20354 catalog_manager.cc:4841] TS 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327): tablet f9aeacbf0b7a4faf8da033d33a15fbbf (table test-table [id=2eb2c55341e544f6912bcfd60064e51b]) successfully deleted
I20260808 23:39:39.250406 20914 raft_consensus.cc:482] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:39.250684 20914 raft_consensus.cc:504] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.253965 20914 leader_election.cc:288] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327), 8ce3da58981442c89ab406c79c737de0 (127.18.186.131:40109)
I20260808 23:39:39.262341 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.253093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:52519 (local address 127.18.186.130:40327)
0808 23:39:39.253285 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.253291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.254754 (+ 1463us) server_negotiation.cc:407] Connection header received
0808 23:39:39.255052 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.255058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.255140 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.255247 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:39.255937 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.256860 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.259750 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.261014 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.261188 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.261271 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.261911 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.262049 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:39:39.262416 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.253494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:54665 (local address 127.18.186.131:40109)
0808 23:39:39.253761 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.253769 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.255009 (+ 1240us) server_negotiation.cc:407] Connection header received
0808 23:39:39.255295 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.255301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.255395 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.255511 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:39.256238 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.257363 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.260030 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.261003 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.261219 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.261312 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.261791 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.261946 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 23:39:39.262467 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.253824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:40109 (local address 127.18.186.129:54665)
0808 23:39:39.254904 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.254922 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.255121 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.255534 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.255539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.255572 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:39.256046 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.256060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.257520 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.257529 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.259862 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.259873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.261264 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.261271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.261479 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.261531 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.261729 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.261887 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":23,"thread_start_us":791,"threads_started":1}
I20260808 23:39:39.263271 20833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2053abe6a8fe4503836aeb95a53cd39e" candidate_uuid: "119d15f2f32140e98db53151fd4ecae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce3da58981442c89ab406c79c737de0" is_pre_election: true
I20260808 23:39:39.263520 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.253480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:40327 (local address 127.18.186.129:52519)
0808 23:39:39.254629 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.254655 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.254892 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.255281 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.255286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.255313 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:39.255713 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.255727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.257009 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.257017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.259560 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.259577 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.261440 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.261449 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.261715 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.261754 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.262957 (+ 1203us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.263093 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":25,"thread_start_us":566,"threads_started":1}
I20260808 23:39:39.263670 20833 raft_consensus.cc:2454] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 119d15f2f32140e98db53151fd4ecae0 in term 0.
I20260808 23:39:39.264206 20461 leader_election.cc:302] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119d15f2f32140e98db53151fd4ecae0, 8ce3da58981442c89ab406c79c737de0; no voters:
I20260808 23:39:39.264742 20914 raft_consensus.cc:2781] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:39.264912 20914 raft_consensus.cc:482] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:39.265039 20914 raft_consensus.cc:3037] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.264230 20691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2053abe6a8fe4503836aeb95a53cd39e" candidate_uuid: "119d15f2f32140e98db53151fd4ecae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619f0b12894c46eab2c1cf90aeeae343" is_pre_election: true
I20260808 23:39:39.267557 20691 raft_consensus.cc:2454] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 119d15f2f32140e98db53151fd4ecae0 in term 0.
I20260808 23:39:39.267760 20914 raft_consensus.cc:504] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.268551 20914 leader_election.cc:288] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [CANDIDATE]: Term 1 election: Requested vote from peers 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327), 8ce3da58981442c89ab406c79c737de0 (127.18.186.131:40109)
I20260808 23:39:39.269223 20833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2053abe6a8fe4503836aeb95a53cd39e" candidate_uuid: "119d15f2f32140e98db53151fd4ecae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce3da58981442c89ab406c79c737de0"
I20260808 23:39:39.269486 20833 raft_consensus.cc:3037] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.272126 20833 raft_consensus.cc:2454] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 119d15f2f32140e98db53151fd4ecae0 in term 1.
I20260808 23:39:39.272608 20461 leader_election.cc:302] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119d15f2f32140e98db53151fd4ecae0, 8ce3da58981442c89ab406c79c737de0; no voters:
I20260808 23:39:39.272919 20914 raft_consensus.cc:2781] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:39.275908 20691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2053abe6a8fe4503836aeb95a53cd39e" candidate_uuid: "119d15f2f32140e98db53151fd4ecae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619f0b12894c46eab2c1cf90aeeae343"
I20260808 23:39:39.276108 20691 raft_consensus.cc:3037] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.279572 20691 raft_consensus.cc:2454] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 119d15f2f32140e98db53151fd4ecae0 in term 1.
I20260808 23:39:39.280447 20914 raft_consensus.cc:686] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 1 LEADER]: Becoming Leader. State: Replica: 119d15f2f32140e98db53151fd4ecae0, State: Running, Role: LEADER
I20260808 23:39:39.280907 20914 consensus_queue.cc:237] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.285238 20368 catalog_manager.cc:5495] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 119d15f2f32140e98db53151fd4ecae0 (127.18.186.129). New cstate: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } health_report { overall_health: UNKNOWN } } }
Aug 08 23:39:39 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:39.339495 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.322288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43343 (local address 127.0.0.1:53494)
0808 23:39:39.323051 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.323074 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.323205 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.323600 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.323604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.324346 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:39.324668 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.324678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.326155 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.326159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.327282 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.327296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.328376 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.328382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.328521 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.329332 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:39.329357 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:39.332010 (+ 2653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:39.336211 (+ 4201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:39.336225 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:39.336233 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:39.336828 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:39.337288 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:39.337293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:39.337297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:39.337484 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:39.338159 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:39.338168 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:39.338372 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.338565 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.338931 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":418,"threads_started":1}
I20260808 23:39:39.339493 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.322152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.18.186.190:43343)
0808 23:39:39.322410 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.322420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.323167 (+ 747us) server_negotiation.cc:407] Connection header received
0808 23:39:39.323347 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.323354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.323462 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.323577 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:39.324836 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.325990 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.327466 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.328164 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.328286 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.332241 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:39.332248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:39.332253 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:39.332293 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:39.335974 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:39.336973 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:39.336979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:39.336981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:39.337154 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:39.337602 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:39.337606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:39.337608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:39.338002 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:39.338121 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.338707 (+ 586us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.339088 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":57}
I20260808 23:39:39.346125 20368 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53494:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 23:39:39.347576 20368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:39.355134 20512 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3861c0769040389678c76199616ccd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ead94fddd81b4fc3a92156810156e777]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:39:39.355888 20512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3861c0769040389678c76199616ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:39.356030 20812 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3861c0769040389678c76199616ccd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ead94fddd81b4fc3a92156810156e777]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:39:39.356612 20812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3861c0769040389678c76199616ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:39.360140 20665 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3861c0769040389678c76199616ccd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ead94fddd81b4fc3a92156810156e777]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:39:39.360702 20665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3861c0769040389678c76199616ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:39.363760 20908 tablet_bootstrap.cc:492] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: Bootstrap starting.
I20260808 23:39:39.365615 20908 tablet_bootstrap.cc:654] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:39.366442 20909 tablet_bootstrap.cc:492] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: Bootstrap starting.
I20260808 23:39:39.368599 20909 tablet_bootstrap.cc:654] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:39.369464 20908 tablet_bootstrap.cc:492] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: No bootstrap required, opened a new log
I20260808 23:39:39.369666 20908 ts_tablet_manager.cc:1397] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260808 23:39:39.370613 20908 raft_consensus.cc:348] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.370960 20908 raft_consensus.cc:374] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:39.371045 20908 raft_consensus.cc:729] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119d15f2f32140e98db53151fd4ecae0, State: Initialized, Role: FOLLOWER
I20260808 23:39:39.371376 20908 consensus_queue.cc:260] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.372004 20914 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.372231 20907 tablet_bootstrap.cc:492] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: Bootstrap starting.
I20260808 23:39:39.372349 20914 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:39.373871 20907 tablet_bootstrap.cc:654] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:39.376544 20908 ts_tablet_manager.cc:1428] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260808 23:39:39.381057 20907 tablet_bootstrap.cc:492] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: No bootstrap required, opened a new log
I20260808 23:39:39.381106 20909 tablet_bootstrap.cc:492] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: No bootstrap required, opened a new log
I20260808 23:39:39.381263 20907 ts_tablet_manager.cc:1397] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 23:39:39.381273 20909 ts_tablet_manager.cc:1397] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.011s
I20260808 23:39:39.381870 20907 raft_consensus.cc:348] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.382088 20907 raft_consensus.cc:374] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:39.382190 20907 raft_consensus.cc:729] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619f0b12894c46eab2c1cf90aeeae343, State: Initialized, Role: FOLLOWER
I20260808 23:39:39.382102 20909 raft_consensus.cc:348] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.382380 20909 raft_consensus.cc:374] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:39.382477 20909 raft_consensus.cc:729] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce3da58981442c89ab406c79c737de0, State: Initialized, Role: FOLLOWER
I20260808 23:39:39.382493 20907 consensus_queue.cc:260] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.382707 20909 consensus_queue.cc:260] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.383104 20907 ts_tablet_manager.cc:1428] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:39:39.382934 20913 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.383231 20913 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:39.383270 20909 ts_tablet_manager.cc:1428] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 23:39:39.383103 20915 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.383948 20915 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:39.661070 20914 consensus_queue.cc:1035] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:39.666771 20940 consensus_queue.cc:1035] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:39.721910 20940 raft_consensus.cc:482] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:39.722150 20940 raft_consensus.cc:504] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.722844 20940 leader_election.cc:288] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327), 8ce3da58981442c89ab406c79c737de0 (127.18.186.131:40109)
I20260808 23:39:39.723560 20691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3861c0769040389678c76199616ccd" candidate_uuid: "119d15f2f32140e98db53151fd4ecae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619f0b12894c46eab2c1cf90aeeae343" is_pre_election: true
I20260808 23:39:39.723562 20833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3861c0769040389678c76199616ccd" candidate_uuid: "119d15f2f32140e98db53151fd4ecae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce3da58981442c89ab406c79c737de0" is_pre_election: true
I20260808 23:39:39.723815 20833 raft_consensus.cc:2454] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 119d15f2f32140e98db53151fd4ecae0 in term 0.
I20260808 23:39:39.723814 20691 raft_consensus.cc:2454] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 119d15f2f32140e98db53151fd4ecae0 in term 0.
I20260808 23:39:39.724341 20461 leader_election.cc:302] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 119d15f2f32140e98db53151fd4ecae0, 619f0b12894c46eab2c1cf90aeeae343, 8ce3da58981442c89ab406c79c737de0; no voters:
I20260808 23:39:39.724653 20940 raft_consensus.cc:2781] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:39.724781 20940 raft_consensus.cc:482] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:39.724908 20940 raft_consensus.cc:3037] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.727555 20940 raft_consensus.cc:504] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.728144 20940 leader_election.cc:288] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [CANDIDATE]: Term 1 election: Requested vote from peers 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327), 8ce3da58981442c89ab406c79c737de0 (127.18.186.131:40109)
I20260808 23:39:39.728636 20691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3861c0769040389678c76199616ccd" candidate_uuid: "119d15f2f32140e98db53151fd4ecae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619f0b12894c46eab2c1cf90aeeae343"
I20260808 23:39:39.728931 20691 raft_consensus.cc:3037] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.729085 20833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3861c0769040389678c76199616ccd" candidate_uuid: "119d15f2f32140e98db53151fd4ecae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce3da58981442c89ab406c79c737de0"
I20260808 23:39:39.729313 20833 raft_consensus.cc:3037] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:39.731901 20691 raft_consensus.cc:2454] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 119d15f2f32140e98db53151fd4ecae0 in term 1.
I20260808 23:39:39.732244 20833 raft_consensus.cc:2454] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 119d15f2f32140e98db53151fd4ecae0 in term 1.
I20260808 23:39:39.732332 20460 leader_election.cc:302] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119d15f2f32140e98db53151fd4ecae0, 619f0b12894c46eab2c1cf90aeeae343; no voters:
I20260808 23:39:39.732688 20940 raft_consensus.cc:2781] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:39.732872 20940 raft_consensus.cc:686] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [term 1 LEADER]: Becoming Leader. State: Replica: 119d15f2f32140e98db53151fd4ecae0, State: Running, Role: LEADER
I20260808 23:39:39.733129 20940 consensus_queue.cc:237] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } }
I20260808 23:39:39.734802 20914 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: TxnStatusTablet state changed. Reason: New leader 119d15f2f32140e98db53151fd4ecae0. Latest consensus state: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.735033 20914 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:39:39.735754 20959 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 23:39:39.736047 20959 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 23:39:39.736356 20366 catalog_manager.cc:5495] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 119d15f2f32140e98db53151fd4ecae0 (127.18.186.129). New cstate: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } health_report { overall_health: UNKNOWN } } }
I20260808 23:39:39.830842 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.823674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36789 (local address 127.0.0.1:58548)
0808 23:39:39.823934 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.823954 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.824124 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.825082 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.825085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.825095 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:39:39.825252 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.825259 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.826796 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.826799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.827936 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.827948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.828943 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.828948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.829155 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.829240 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:39:39.830304 (+ 1064us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:39:39.830323 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.830375 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.830527 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 23:39:39.830859 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.823851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.18.186.129:36789)
0808 23:39:39.824611 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.824618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.824639 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:39:39.824743 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.824749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.824875 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.825040 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:39.825407 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.826661 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.828074 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.828775 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.828903 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.829459 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:39.830197 (+ 738us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:39.830348 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.830437 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.830535 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":425,"threads_started":1}
I20260808 23:39:39.835758 20691 raft_consensus.cc:1261] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Refusing update from remote peer 119d15f2f32140e98db53151fd4ecae0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:39.835839 20833 raft_consensus.cc:1261] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0 [term 1 FOLLOWER]: Refusing update from remote peer 119d15f2f32140e98db53151fd4ecae0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:39.836509 20940 consensus_queue.cc:1035] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:39.836881 20914 consensus_queue.cc:1035] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:39.839671 20913 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: TxnStatusTablet state changed. Reason: New leader 119d15f2f32140e98db53151fd4ecae0. Latest consensus state: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.839936 20913 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:39.842440 20955 mvcc.cc:205] Tried to move back new op lower bound from 7316407827800973312 to 7316407827389186048. Current Snapshot: MvccSnapshot[applied={T|T < 7316407827800973312}]
I20260808 23:39:39.844369 20915 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: TxnStatusTablet state changed. Reason: New leader 119d15f2f32140e98db53151fd4ecae0. Latest consensus state: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.844775 20915 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:39.847177 20914 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.847470 20914 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:39:39.849210 20914 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.849573 20914 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 119d15f2f32140e98db53151fd4ecae0: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:39:39.852420 20915 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.852730 20915 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 8ce3da58981442c89ab406c79c737de0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:39.854490 20913 tablet_replica.cc:439] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "119d15f2f32140e98db53151fd4ecae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119d15f2f32140e98db53151fd4ecae0" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36789 } } peers { permanent_uuid: "619f0b12894c46eab2c1cf90aeeae343" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40327 } } peers { permanent_uuid: "8ce3da58981442c89ab406c79c737de0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40109 } } }
I20260808 23:39:39.854763 20913 tablet_replica.cc:442] T 2f3861c0769040389678c76199616ccd P 619f0b12894c46eab2c1cf90aeeae343: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:39.867144 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.860760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.18.186.129:36789)
0808 23:39:39.860999 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.861008 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.861481 (+ 473us) server_negotiation.cc:407] Connection header received
0808 23:39:39.861695 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.861704 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.861769 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.861842 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:39.862312 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.863167 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.865331 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.865770 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.865861 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.866341 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:39.866459 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:39.866522 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.866722 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.866788 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:39:39.885200 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.878645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:43367 (local address 127.18.186.129:36789)
0808 23:39:39.878899 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.878909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.879693 (+ 784us) server_negotiation.cc:407] Connection header received
0808 23:39:39.879855 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.879858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.879943 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.880015 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:39.880464 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.881259 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.883270 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.883894 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.883997 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.884499 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:39.884600 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:39.884664 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.884880 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.884940 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:39:39.885360 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.878536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36789 (local address 127.18.186.129:43367)
0808 23:39:39.879548 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:39.879569 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:39.879742 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:39.880087 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:39.880092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:39.880105 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:39:39.880334 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.880345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.881390 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.881394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.883118 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:39.883133 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.884049 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.884055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:39.884233 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.884318 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:39:39.884726 (+ 408us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:39:39.884737 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 23:39:39.884837 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 23:39:39.884983 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":595,"threads_started":1}
I20260808 23:39:39.951910 20367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:39.952157 20367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:39.955250 20367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41c9658cd06d4015a96910a61e2cf0ab: Sending DeleteTablet for 3 replicas of tablet 2053abe6a8fe4503836aeb95a53cd39e
I20260808 23:39:39.956029 20513 tablet_service.cc:1514] Processing DeleteTablet for tablet 2053abe6a8fe4503836aeb95a53cd39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:39 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:58532
I20260808 23:39:39.956128 20665 tablet_service.cc:1514] Processing DeleteTablet for tablet 2053abe6a8fe4503836aeb95a53cd39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:39 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53614
I20260808 23:39:39.956575 20812 tablet_service.cc:1514] Processing DeleteTablet for tablet 2053abe6a8fe4503836aeb95a53cd39e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:39 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:35138
I20260808 23:39:39.957239 20978 tablet_replica.cc:330] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: stopping tablet replica
I20260808 23:39:39.957607 20978 raft_consensus.cc:2229] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:39.958233 20978 raft_consensus.cc:2258] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:39.960469 20978 ts_tablet_manager.cc:1905] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:39.969967 20978 ts_tablet_manager.cc:1918] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:39.970201 20978 log.cc:1196] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/2053abe6a8fe4503836aeb95a53cd39e
I20260808 23:39:39.970876 20978 ts_tablet_manager.cc:1939] T 2053abe6a8fe4503836aeb95a53cd39e P 8ce3da58981442c89ab406c79c737de0: Deleting consensus metadata
I20260808 23:39:39.972277 20355 catalog_manager.cc:4841] TS 8ce3da58981442c89ab406c79c737de0 (127.18.186.131:40109): tablet 2053abe6a8fe4503836aeb95a53cd39e (table test-table [id=a228fcb48f444c949bdcd499ad98423c]) successfully deleted
I20260808 23:39:39.976220 20977 tablet_replica.cc:330] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: stopping tablet replica
I20260808 23:39:39.976568 20977 raft_consensus.cc:2229] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:39.976899 20977 raft_consensus.cc:2258] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:39.977636 20976 tablet_replica.cc:330] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: stopping tablet replica
I20260808 23:39:39.977960 20976 raft_consensus.cc:2229] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:39:39.978150 20977 ts_tablet_manager.cc:1905] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:39.978932 20976 raft_consensus.cc:2258] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:39.980288 20976 ts_tablet_manager.cc:1905] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:39.986476 20977 ts_tablet_manager.cc:1918] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:39.986696 20977 log.cc:1196] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/2053abe6a8fe4503836aeb95a53cd39e
I20260808 23:39:39.987257 20977 ts_tablet_manager.cc:1939] T 2053abe6a8fe4503836aeb95a53cd39e P 619f0b12894c46eab2c1cf90aeeae343: Deleting consensus metadata
I20260808 23:39:39.987717 20976 ts_tablet_manager.cc:1918] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:39.987860 20976 log.cc:1196] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/2053abe6a8fe4503836aeb95a53cd39e
I20260808 23:39:39.988358 20976 ts_tablet_manager.cc:1939] T 2053abe6a8fe4503836aeb95a53cd39e P 119d15f2f32140e98db53151fd4ecae0: Deleting consensus metadata
I20260808 23:39:39.988669 20354 catalog_manager.cc:4841] TS 619f0b12894c46eab2c1cf90aeeae343 (127.18.186.130:40327): tablet 2053abe6a8fe4503836aeb95a53cd39e (table test-table [id=a228fcb48f444c949bdcd499ad98423c]) successfully deleted
I20260808 23:39:39.989478 20353 catalog_manager.cc:4841] TS 119d15f2f32140e98db53151fd4ecae0 (127.18.186.129:36789): tablet 2053abe6a8fe4503836aeb95a53cd39e (table test-table [id=a228fcb48f444c949bdcd499ad98423c]) successfully deleted
Aug 08 23:39:39 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:39:39.997570 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:39.976792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58576 (local address 127.18.186.129:36789)
0808 23:39:39.977011 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:39.977019 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:39.980072 (+ 3053us) server_negotiation.cc:407] Connection header received
0808 23:39:39.980280 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:39.980286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:39.980394 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:39.980501 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:39.982607 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.983740 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.985666 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:39.986452 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:39.986564 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:39.991628 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:39.991652 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:39.991667 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:39.991698 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:39.994708 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:39.995400 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:39.995408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:39.995410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:39.995661 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:39.996053 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:39.996059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:39.996062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:39.996402 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:39.996787 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:39.996974 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 23:39:39.997198 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 23:39:39.998359 20582 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58576
I20260808 23:39:40.016106 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:40.001479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.18.186.190:43343)
0808 23:39:40.002439 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:40.002446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:40.002467 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:39:40.002527 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:40.002532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:40.002611 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:40.002745 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:40.004212 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:40.005325 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:40.006505 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:40.007135 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:40.007254 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:40.009633 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:40.009641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:40.009647 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:40.009692 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:40.013251 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:40.013967 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:40.013975 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:40.013979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:40.014129 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:40.014556 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:40.014562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:40.014572 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:40.014920 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:40.015176 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:40.015521 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 23:39:40.015748 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":660,"threads_started":1}
W20260808 23:39:40.016675 20367 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53502
I20260808 23:39:40.019492 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20442
I20260808 23:39:40.033177 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:40.020494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58584 (local address 127.18.186.129:36789)
0808 23:39:40.020693 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:40.020699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:40.020893 (+ 194us) server_negotiation.cc:407] Connection header received
0808 23:39:40.021058 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:40.021064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:40.021149 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:40.021269 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:40.022349 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:40.023611 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:40.024809 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:40.025271 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:40.025439 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:40.027618 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:40.027629 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:40.027634 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:40.027665 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:40.030489 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:40.031125 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:40.031131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:40.031134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:40.031302 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:40.031692 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:40.031698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:40.031701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:40.031994 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:40.032291 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:40.032579 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 23:39:40.032803 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 23:39:40.034049 20582 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58584
W20260808 23:39:40.034631 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20442: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:40.034823 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 20442
I20260808 23:39:40.058292 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20592
Aug 08 23:39:40 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:39:40.073081 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:40.059109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53624 (local address 127.18.186.130:40327)
0808 23:39:40.059672 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:40.059676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:40.059692 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:39:40.059747 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:40.059750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:40.059809 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:40.059906 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:40.061067 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:40.061795 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:40.063129 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:40.063685 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:40.063777 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:40.067679 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:40.067691 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:40.067706 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:40.067741 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:40.070592 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:40.071175 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:40.071182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:40.071184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:40.071320 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:40.071848 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:40.071852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:40.071854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:40.072191 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:40.072431 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:40.072654 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 23:39:40.072814 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":323,"threads_started":1}
W20260808 23:39:40.073807 20733 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53624
W20260808 23:39:40.074467 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20592: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:40.074640 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 20592
I20260808 23:39:40.093938 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20743
Aug 08 23:39:40 dist-test-slave-5k54 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:39:40.110412 20987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:40.094660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35150 (local address 127.18.186.131:40109)
0808 23:39:40.095191 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:40.095196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:40.095210 (+ 14us) server_negotiation.cc:407] Connection header received
0808 23:39:40.095260 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:40.095264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:40.095326 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:40.095472 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:40.096755 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:40.097487 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:40.098748 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:40.099169 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:40.099311 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:40.103771 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:40.103796 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:40.103813 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:40.103850 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:40.107367 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:40.108151 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:40.108159 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:40.108162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:40.108297 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:40.108692 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:40.108699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:40.108702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:40.109098 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:40.109433 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:40.109790 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 23:39:40.110045 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":261,"threads_started":1}
W20260808 23:39:40.111263 20881 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35150
W20260808 23:39:40.111989 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20743: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:40.112155 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 20743
I20260808 23:39:40.135804 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 20336
I20260808 23:39:40.137041 20411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:40.254748 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 20336
2026-08-08T23:39:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4948 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:39:40 dist-test-slave-5k54 krb5kdc[20992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:39:40 dist-test-slave-5k54 krb5kdc[20992](info): set up 2 sockets
Aug 08 23:39:40 dist-test-slave-5k54 krb5kdc[20992](info): commencing operation
krb5kdc: starting...
W20260808 23:39:41.430073 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:39:41 dist-test-slave-5k54 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:39:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:39:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:39:41.636869 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44393
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:42171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:44393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:41.967010 21006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:41.967324 21006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:41.967464 21006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:41.974764 21006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:39:41.974879 21006 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 23:39:41.974948 21006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:41.975049 21006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:41.975116 21006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:39:41.975179 21006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:39:41.985075 21006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:42171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:44393
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:41.986418 21006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:41.987732 21006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:41.999447 21015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:41.999478 21013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:41.999367 21012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:42.000365 21006 server_base.cc:1034] running on GCE node
I20260808 23:39:42.001205 21006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:42.002676 21006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:42.003955 21006 hybrid_clock.cc:648] HybridClock initialized: now 1786232382003882 us; error 47 us; skew 500 ppm
I20260808 23:39:42.004330 21006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:42 dist-test-slave-5k54 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232382, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:42.007692 21006 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:39:42.009192 21006 webserver.cc:460] Webserver started at http://127.18.186.190:34435/ using document root <none> and password file <none>
I20260808 23:39:42.009932 21006 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:42.010070 21006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:42.010408 21006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:42.013471 21006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "d9b8f648582844b09d563f1ac1cfa096"
format_stamp: "Formatted at 2026-08-08 23:39:42 on dist-test-slave-5k54"
server_key: "d958234c0d749aabffdd61689acbc907"
server_key_iv: "ed51f33f05012c0cae820fb093baac9e"
server_key_version: "encryptionkey@0"
I20260808 23:39:42.014281 21006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "d9b8f648582844b09d563f1ac1cfa096"
format_stamp: "Formatted at 2026-08-08 23:39:42 on dist-test-slave-5k54"
server_key: "d958234c0d749aabffdd61689acbc907"
server_key_iv: "ed51f33f05012c0cae820fb093baac9e"
server_key_version: "encryptionkey@0"
I20260808 23:39:42.019519 21006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 23:39:42.022918 21022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:42.024420 21006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:39:42.024614 21006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "d9b8f648582844b09d563f1ac1cfa096"
format_stamp: "Formatted at 2026-08-08 23:39:42 on dist-test-slave-5k54"
server_key: "d958234c0d749aabffdd61689acbc907"
server_key_iv: "ed51f33f05012c0cae820fb093baac9e"
server_key_version: "encryptionkey@0"
I20260808 23:39:42.024853 21006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:42.046255 21006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:42.051319 21006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:42.051623 21006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:42.072790 21006 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:44393
I20260808 23:39:42.072798 21087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:44393 every 8 connection(s)
I20260808 23:39:42.074152 21006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:39:42.077108 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 21006
I20260808 23:39:42.077316 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:39:42.078167 19178 external_mini_cluster.cc:1428] Setting key f3720966275eb081d5f74b42b0e1e32d
I20260808 23:39:42.078408 21088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:42.092905 21088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: Bootstrap starting.
Aug 08 23:39:42 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:42.096107 21088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:42.097159 21088 log.cc:824] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:42.099844 21088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: No bootstrap required, opened a new log
I20260808 23:39:42.101598 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:42.080391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.18.186.190:44393)
0808 23:39:42.081165 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:42.081177 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:42.081220 (+ 43us) server_negotiation.cc:407] Connection header received
0808 23:39:42.083103 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:42.083124 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:42.083586 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:42.083926 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:42.085710 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.086976 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.088737 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.089441 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.089584 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:42.094328 (+ 4744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:42.094336 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:42.094352 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:42.094401 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:42.097580 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:42.098278 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:42.098284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:42.098290 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:42.098456 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:42.098831 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:42.098835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:42.098837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:42.099396 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:42.099784 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:42.100179 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 23:39:42.100617 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":378,"threads_started":1}
I20260808 23:39:42.108067 21088 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8f648582844b09d563f1ac1cfa096" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44393 } }
I20260808 23:39:42.108484 21088 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:42.108597 21088 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b8f648582844b09d563f1ac1cfa096, State: Initialized, Role: FOLLOWER
I20260808 23:39:42.109205 21088 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8f648582844b09d563f1ac1cfa096" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44393 } }
I20260808 23:39:42.109439 21088 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:39:42.109575 21088 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:39:42.109736 21088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:42.112257 21088 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8f648582844b09d563f1ac1cfa096" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44393 } }
I20260808 23:39:42.112736 21088 leader_election.cc:302] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9b8f648582844b09d563f1ac1cfa096; no voters:
I20260808 23:39:42.113288 21088 leader_election.cc:288] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:39:42.113428 21094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:42.114130 21094 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [term 1 LEADER]: Becoming Leader. State: Replica: d9b8f648582844b09d563f1ac1cfa096, State: Running, Role: LEADER
I20260808 23:39:42.114668 21094 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8f648582844b09d563f1ac1cfa096" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44393 } }
I20260808 23:39:42.115317 21088 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:39:42.117457 21096 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b8f648582844b09d563f1ac1cfa096. Latest consensus state: current_term: 1 leader_uuid: "d9b8f648582844b09d563f1ac1cfa096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8f648582844b09d563f1ac1cfa096" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44393 } } }
I20260808 23:39:42.117868 21096 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:42.117713 21095 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9b8f648582844b09d563f1ac1cfa096" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8f648582844b09d563f1ac1cfa096" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44393 } } }
I20260808 23:39:42.117980 21095 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:42.119665 21103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:39:42.124028 21103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:39:42.132261 21103 catalog_manager.cc:1347] Generated new cluster ID: d4ffa8a3ecc44956ae50b50bb3801f0b
I20260808 23:39:42.132395 21103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:39:42.166795 21103 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:39:42.168130 21103 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:39:42.177733 21103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: Generated new TSK 0
I20260808 23:39:42.178570 21103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:39:42.233743 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44393
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:42171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:42.558713 21117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:42.559036 21117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:42.559212 21117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:42.566361 21117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:42.566514 21117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:42.566677 21117 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 23:39:42.566762 21117 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:39:42.566883 21117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:39:42.576484 21117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:42171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:44393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:42.577862 21117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:42.579186 21117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:42.592072 21123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:42.592141 21124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:42.592170 21126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:42.592938 21117 server_base.cc:1034] running on GCE node
I20260808 23:39:42.593501 21117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:42.594319 21117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:42.595521 21117 hybrid_clock.cc:648] HybridClock initialized: now 1786232382595479 us; error 28 us; skew 500 ppm
I20260808 23:39:42.595844 21117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:42 dist-test-slave-5k54 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232382, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:42.599009 21117 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:39:42.600456 21117 webserver.cc:460] Webserver started at http://127.18.186.129:33407/ using document root <none> and password file <none>
I20260808 23:39:42.601190 21117 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:42.601336 21117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:42.601652 21117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:42.604683 21117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "02be3648e7bc4d2c969421881aa48379"
format_stamp: "Formatted at 2026-08-08 23:39:42 on dist-test-slave-5k54"
server_key: "02e08fee2be07831781a74c0942c5415"
server_key_iv: "19c6feba9dedeee44e5fab3c4de0fc23"
server_key_version: "encryptionkey@0"
I20260808 23:39:42.605443 21117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "02be3648e7bc4d2c969421881aa48379"
format_stamp: "Formatted at 2026-08-08 23:39:42 on dist-test-slave-5k54"
server_key: "02e08fee2be07831781a74c0942c5415"
server_key_iv: "19c6feba9dedeee44e5fab3c4de0fc23"
server_key_version: "encryptionkey@0"
I20260808 23:39:42.610491 21117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 23:39:42.613845 21133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:42.615118 21117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:39:42.615293 21117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "02be3648e7bc4d2c969421881aa48379"
format_stamp: "Formatted at 2026-08-08 23:39:42 on dist-test-slave-5k54"
server_key: "02e08fee2be07831781a74c0942c5415"
server_key_iv: "19c6feba9dedeee44e5fab3c4de0fc23"
server_key_version: "encryptionkey@0"
I20260808 23:39:42.615578 21117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:42.640398 21117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:42.644897 21117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:42.645196 21117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:42.648051 21117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:42.648180 21117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:42.648310 21117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:42.648403 21117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:39:42 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232382, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:42.671911 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:42.648746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:55373 (local address 127.18.186.190:44393)
0808 23:39:42.649396 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:42.649401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:42.650497 (+ 1096us) server_negotiation.cc:407] Connection header received
0808 23:39:42.652598 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:42.652604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:42.652695 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:42.652823 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:42.655162 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.656123 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.658596 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.659070 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.659206 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:42.663633 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:42.663643 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:42.663652 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:42.663713 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:42.667901 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:42.669046 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:42.669051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:42.669053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:42.669173 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:42.669854 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:42.669860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:42.669864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:42.670170 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:42.670341 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:42.671239 (+ 898us) server_negotiation.cc:299] Negotiation successful
0808 23:39:42.671518 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":379,"threads_started":1}
I20260808 23:39:42.673044 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:42.649478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44393 (local address 127.18.186.129:55373)
0808 23:39:42.650344 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:42.650385 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:42.652325 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:42.653058 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:42.653067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:42.653999 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:42.654976 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:42.654996 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.656296 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.656310 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:42.658441 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:42.658452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.659327 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.659335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:42.659632 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:42.660454 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:42.660478 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:42.663279 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:42.668119 (+ 4840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:42.668132 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:42.668154 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:42.668858 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:42.669351 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:42.669358 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:42.669364 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:42.669701 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:42.670312 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:42.670321 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:42.670857 (+ 536us) client_negotiation.cc:769] Sending connection context
0808 23:39:42.671153 (+ 296us) client_negotiation.cc:240] Negotiation successful
0808 23:39:42.671798 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":378,"threads_started":1}
I20260808 23:39:42.695765 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:42.678734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:43809 (local address 127.18.186.190:44393)
0808 23:39:42.678924 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:42.678931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:42.679880 (+ 949us) server_negotiation.cc:407] Connection header received
0808 23:39:42.680078 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:42.680086 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:42.680182 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:42.680289 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:42.681784 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.682493 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.685345 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.685904 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.686062 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:42.688899 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:42.688904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:42.688907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:42.688939 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:42.692391 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:42.693528 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:42.693537 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:42.693541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:42.693722 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:42.694134 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:42.694141 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:42.694146 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:42.694536 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:42.694690 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:42.695145 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 23:39:42.695391 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:39:42.695915 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:42.678934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44393 (local address 127.18.186.129:43809)
0808 23:39:42.679776 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:42.679796 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:42.679940 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:42.680361 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:42.680373 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:42.681083 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:42.681569 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:42.681586 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.682790 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.682799 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:42.685149 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:42.685176 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:42.686148 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:42.686159 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:42.686474 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:42.687208 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:42.687239 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:42.688672 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:42.692571 (+ 3899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:42.692580 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:42.692587 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:42.693370 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:42.693836 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:42.693843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:42.693848 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:42.694010 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:42.694658 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:42.694663 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:42.694851 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 23:39:42.694998 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 23:39:42.695400 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":421,"threads_started":1}
I20260808 23:39:42.709192 21117 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:36417
I20260808 23:39:42.709224 21261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:36417 every 8 connection(s)
I20260808 23:39:42.710440 21117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:39:42.712549 21262 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44393
I20260808 23:39:42.712746 21262 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:42.713382 21262 heartbeater.cc:511] Master 127.18.186.190:44393 requested a full tablet report, sending...
I20260808 23:39:42.714679 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 21117
I20260808 23:39:42.714905 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:39:42.715668 21041 ts_manager.cc:194] Registered new tserver with Master: 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417)
I20260808 23:39:42.715833 19178 external_mini_cluster.cc:1428] Setting key 28caa5c401ca521b52305eeabe067e3f
I20260808 23:39:42.717415 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:55373
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:39:42.759040 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44393
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:42171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:43.083739 21266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:43.084060 21266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:43.084225 21266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:43.091547 21266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:43.091696 21266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:43.091902 21266 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 23:39:43.091993 21266 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:39:43.092126 21266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:39:43.101722 21266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:42171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:44393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:43.103039 21266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:43.104379 21266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:43.117446 21274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:43.117487 21271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:43.117518 21272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:43.118595 21266 server_base.cc:1034] running on GCE node
I20260808 23:39:43.119220 21266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:43.120213 21266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:43.121418 21266 hybrid_clock.cc:648] HybridClock initialized: now 1786232383121380 us; error 38 us; skew 500 ppm
I20260808 23:39:43.121752 21266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:43 dist-test-slave-5k54 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:43.125072 21266 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:39:43.126592 21266 webserver.cc:460] Webserver started at http://127.18.186.130:38539/ using document root <none> and password file <none>
I20260808 23:39:43.127267 21266 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:43.127410 21266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:43.127791 21266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:43.130806 21266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "39273e56795c47b4b9dd0fb5cbedd7bf"
format_stamp: "Formatted at 2026-08-08 23:39:43 on dist-test-slave-5k54"
server_key: "10851b1ea19979eb11077a282724d4dd"
server_key_iv: "92356c1b40d77b5bd07c3af253c28ce7"
server_key_version: "encryptionkey@0"
I20260808 23:39:43.131583 21266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "39273e56795c47b4b9dd0fb5cbedd7bf"
format_stamp: "Formatted at 2026-08-08 23:39:43 on dist-test-slave-5k54"
server_key: "10851b1ea19979eb11077a282724d4dd"
server_key_iv: "92356c1b40d77b5bd07c3af253c28ce7"
server_key_version: "encryptionkey@0"
I20260808 23:39:43.136641 21266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 23:39:43.140136 21282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:43.141448 21266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:39:43.141610 21266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "39273e56795c47b4b9dd0fb5cbedd7bf"
format_stamp: "Formatted at 2026-08-08 23:39:43 on dist-test-slave-5k54"
server_key: "10851b1ea19979eb11077a282724d4dd"
server_key_iv: "92356c1b40d77b5bd07c3af253c28ce7"
server_key_version: "encryptionkey@0"
I20260808 23:39:43.141825 21266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:43.170169 21266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:43.174310 21266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:43.174610 21266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:43.177477 21266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:43.177603 21266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:43.177736 21266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:43.178261 21266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
Aug 08 23:39:43 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:43.201051 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.177950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:37403 (local address 127.18.186.190:44393)
0808 23:39:43.178141 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:43.178146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:43.179722 (+ 1576us) server_negotiation.cc:407] Connection header received
0808 23:39:43.181773 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:43.181779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:43.181872 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:43.181998 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:43.183914 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.185002 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.187517 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.187980 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.188156 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.193003 (+ 4847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:43.193012 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:43.193019 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:43.193071 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:43.197114 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.198157 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.198165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.198169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.198336 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.198933 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.198937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.198940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.199334 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:43.199522 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:43.200461 (+ 939us) server_negotiation.cc:299] Negotiation successful
0808 23:39:43.200681 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:39:43.202070 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.178607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44393 (local address 127.18.186.130:37403)
0808 23:39:43.179554 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:43.179597 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:43.181437 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:43.182214 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:43.182222 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:43.182818 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:43.183714 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.183749 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.185150 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.185156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.187284 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.187301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.188277 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.188287 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.188665 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.189441 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:43.189481 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:43.192706 (+ 3225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:43.197273 (+ 4567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:43.197285 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:43.197308 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:43.197987 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:43.198475 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:43.198481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:43.198486 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:43.198794 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:43.199513 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:43.199521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:43.200018 (+ 497us) client_negotiation.cc:769] Sending connection context
0808 23:39:43.200338 (+ 320us) client_negotiation.cc:240] Negotiation successful
0808 23:39:43.200885 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":382,"threads_started":1}
I20260808 23:39:43.222191 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.207533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:45329 (local address 127.18.186.190:44393)
0808 23:39:43.207760 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:43.207768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:43.208354 (+ 586us) server_negotiation.cc:407] Connection header received
0808 23:39:43.208490 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:43.208498 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:43.208608 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:43.208745 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:43.209588 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.210866 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.212493 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.213384 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.213554 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.215163 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:43.215172 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:43.215180 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:43.215227 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:43.219317 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.220131 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.220139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.220143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.220297 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.220678 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.220683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.220686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.221075 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:43.221228 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:43.221622 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 23:39:43.221855 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:39:43.222289 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.207582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44393 (local address 127.18.186.130:45329)
0808 23:39:43.208262 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:43.208277 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:43.208385 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:43.208753 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:43.208756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:43.209163 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:43.209458 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.209467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.211010 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.211015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.212344 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.212356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.213575 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.213579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.213706 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.214285 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:43.214302 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:43.214962 (+ 660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:43.219475 (+ 4513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:43.219480 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:43.219483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:43.220012 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:43.220407 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:43.220413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:43.220417 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:43.220570 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:43.221191 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:43.221196 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:43.221382 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 23:39:43.221513 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 23:39:43.221847 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":376,"threads_started":1}
I20260808 23:39:43.236610 21266 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:41455
I20260808 23:39:43.236651 21410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:41455 every 8 connection(s)
I20260808 23:39:43.237926 21266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:39:43.238499 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 21266
I20260808 23:39:43.238690 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:39:43.239322 19178 external_mini_cluster.cc:1428] Setting key 3aaf31348bb353c13b2d50020d0efef7
I20260808 23:39:43.240474 21411 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44393
I20260808 23:39:43.240720 21411 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:43.241492 21411 heartbeater.cc:511] Master 127.18.186.190:44393 requested a full tablet report, sending...
I20260808 23:39:43.242949 21041 ts_manager.cc:194] Registered new tserver with Master: 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455)
I20260808 23:39:43.243999 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:37403
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:39:43.282845 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44393
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:42171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:43.609382 21415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:43.609650 21415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:43.609848 21415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:43.617223 21415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:43.617345 21415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:43.617499 21415 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 23:39:43.617576 21415 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:39:43.617698 21415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:39:43.627619 21415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:42171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:44393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:43.629019 21415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:43.630403 21415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:43.643882 21424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:43.643918 21421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:43.643793 21422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:43.645125 21415 server_base.cc:1034] running on GCE node
I20260808 23:39:43.645705 21415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:43.646668 21415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:43.647861 21415 hybrid_clock.cc:648] HybridClock initialized: now 1786232383647805 us; error 50 us; skew 500 ppm
I20260808 23:39:43.648145 21415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:43 dist-test-slave-5k54 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:43.651523 21415 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:39:43.653025 21415 webserver.cc:460] Webserver started at http://127.18.186.131:36811/ using document root <none> and password file <none>
I20260808 23:39:43.653712 21415 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:43.653841 21415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:43.654171 21415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:43.657197 21415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "97f1edb32a3448f68d7bc17d70d67e89"
format_stamp: "Formatted at 2026-08-08 23:39:43 on dist-test-slave-5k54"
server_key: "699a85e86401a153feb2111fd74fa778"
server_key_iv: "4d3282bff0c58e7636218f99c6373a1b"
server_key_version: "encryptionkey@0"
I20260808 23:39:43.657977 21415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "97f1edb32a3448f68d7bc17d70d67e89"
format_stamp: "Formatted at 2026-08-08 23:39:43 on dist-test-slave-5k54"
server_key: "699a85e86401a153feb2111fd74fa778"
server_key_iv: "4d3282bff0c58e7636218f99c6373a1b"
server_key_version: "encryptionkey@0"
I20260808 23:39:43.663316 21415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 23:39:43.666836 21433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:43.668237 21415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:39:43.668413 21415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "97f1edb32a3448f68d7bc17d70d67e89"
format_stamp: "Formatted at 2026-08-08 23:39:43 on dist-test-slave-5k54"
server_key: "699a85e86401a153feb2111fd74fa778"
server_key_iv: "4d3282bff0c58e7636218f99c6373a1b"
server_key_version: "encryptionkey@0"
I20260808 23:39:43.668623 21415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:43.703039 21415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:43.707662 21415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:43.707963 21415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:43.710817 21415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:43.710963 21415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:43.711102 21415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:43.711189 21415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:43.720443 21262 heartbeater.cc:503] Master 127.18.186.190:44393 was elected leader, sending a full tablet report...
Aug 08 23:39:43 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:43.739207 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.711759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:54193 (local address 127.18.186.190:44393)
0808 23:39:43.711974 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:43.711980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:43.713875 (+ 1895us) server_negotiation.cc:407] Connection header received
0808 23:39:43.716581 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:43.716588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:43.716681 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:43.716760 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:43.719411 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.720624 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.723223 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.723948 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.724108 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.729553 (+ 5445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:43.729562 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:43.729571 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:43.729633 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:43.733639 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.734596 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.734606 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.734612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.734821 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.735301 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.735308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.735311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.735658 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:43.735830 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:43.738650 (+ 2820us) server_negotiation.cc:299] Negotiation successful
0808 23:39:43.738874 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:39:43.739389 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.712771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44393 (local address 127.18.186.131:54193)
0808 23:39:43.713712 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:43.713758 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:43.716201 (+ 2443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:43.717367 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:43.717382 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:43.718178 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:43.719205 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.719238 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.720845 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.720851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.723068 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.723081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.724249 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.724258 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.724558 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.725365 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:43.725390 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:43.729236 (+ 3846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:43.733808 (+ 4572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:43.733823 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:43.733841 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:43.734444 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:43.734954 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:43.734958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:43.734960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:43.735181 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:43.735853 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:43.735861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:43.736474 (+ 613us) client_negotiation.cc:769] Sending connection context
0808 23:39:43.736886 (+ 412us) client_negotiation.cc:240] Negotiation successful
0808 23:39:43.737576 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":388,"threads_started":1}
I20260808 23:39:43.754863 21415 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:37047
I20260808 23:39:43.754868 21560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:37047 every 8 connection(s)
I20260808 23:39:43.756279 21415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:39:43.758605 21561 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44393
I20260808 23:39:43.758849 21561 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:43.759696 21561 heartbeater.cc:511] Master 127.18.186.190:44393 requested a full tablet report, sending...
I20260808 23:39:43.761281 21041 ts_manager.cc:194] Registered new tserver with Master: 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:43.762009 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 21415
I20260808 23:39:43.762199 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:39:43.762426 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:54193
I20260808 23:39:43.762730 19178 external_mini_cluster.cc:1428] Setting key 43b0afc24e2b8b79d4983b35fd658d52
I20260808 23:39:43.763854 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.747664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:41661 (local address 127.18.186.190:44393)
0808 23:39:43.747875 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:43.747884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:43.749007 (+ 1123us) server_negotiation.cc:407] Connection header received
0808 23:39:43.749230 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:43.749236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:43.749321 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:43.749426 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:43.750785 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.752083 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.754484 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.755134 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.755262 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.757335 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:43.757343 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:43.757350 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:43.757393 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:43.760934 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.761726 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.761733 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.761738 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.761918 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.762335 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.762340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.762343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.762687 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:43.762847 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:43.763228 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 23:39:43.763486 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:39:43.763953 21541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.748042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44393 (local address 127.18.186.131:41661)
0808 23:39:43.748867 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:43.748908 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:43.749087 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:43.749450 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:43.749455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:43.750098 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:43.750583 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.750608 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.752311 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.752320 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.754243 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.754267 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.755301 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.755305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.755467 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.756067 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:43.756084 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:43.757096 (+ 1012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:43.761055 (+ 3959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:43.761060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:43.761076 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:43.761601 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:43.762052 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:43.762057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:43.762061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:43.762203 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:43.762793 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:43.762799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:43.762995 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 23:39:43.763133 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 23:39:43.763483 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":468,"threads_started":1}
I20260808 23:39:43.765857 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:39:43 dist-test-slave-5k54 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:39:43 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:43.815331 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.801938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46170 (local address 127.18.186.190:44393)
0808 23:39:43.802096 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:43.802100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:43.802437 (+ 337us) server_negotiation.cc:407] Connection header received
0808 23:39:43.802588 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:43.802592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:43.802656 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:43.802751 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:43.804264 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.805118 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.806342 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.806780 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.806871 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.810588 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:43.810593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:43.810596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:43.810627 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:43.813084 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.813668 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.813672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.813674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.813796 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:43.814106 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:43.814109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:43.814111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:43.814361 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:43.814549 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:43.814777 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 23:39:43.815015 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:39:43.819481 21040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:39:43.826512 21040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:43.865185 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.839369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.18.186.130:41455)
0808 23:39:43.856929 (+ 17560us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:43.856942 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:43.856981 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:39:43.857112 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:43.857125 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:43.857495 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:43.857748 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:43.858710 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.860153 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.862154 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.863680 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.863846 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.864033 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:43.864688 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 23:39:43.864876 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17333,"thread_start_us":17190,"threads_started":1}
I20260808 23:39:43.865235 21575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.838885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:41455 (local address 127.0.0.1:49876)
0808 23:39:43.855169 (+ 16284us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:43.855189 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:43.855320 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:43.858025 (+ 2705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:43.858032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:43.858062 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:43.858501 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.858513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.860283 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.860291 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.861960 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.861977 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.864165 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.864172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.864382 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.864450 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 23:39:43.864603 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 23:39:43.864808 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16143,"spinlock_wait_cycles":22784,"thread_start_us":11640,"threads_started":1}
I20260808 23:39:43.866248 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.839528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36417 (local address 127.0.0.1:53556)
0808 23:39:43.857746 (+ 18218us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:43.857768 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:43.858113 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:43.859056 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:43.859063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:43.859090 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:43.859708 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.859721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.861613 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.861618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.863330 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.863347 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.865472 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.865478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.865646 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.865684 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 23:39:43.865781 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 23:39:43.865908 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18101,"mutex_wait_us":3,"thread_start_us":17971,"threads_started":1}
I20260808 23:39:43.866346 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.840177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.18.186.129:36417)
0808 23:39:43.857298 (+ 17121us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:43.857311 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:43.858011 (+ 700us) server_negotiation.cc:407] Connection header received
0808 23:39:43.858318 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:43.858327 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:43.858594 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:43.858815 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:43.859934 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.861476 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.863518 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.864853 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.865060 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.865233 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:43.865849 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 23:39:43.866027 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16925,"thread_start_us":16780,"threads_started":1}
I20260808 23:39:43.867478 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.838693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37047 (local address 127.0.0.1:36290)
0808 23:39:43.851092 (+ 12399us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:43.851150 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:43.851405 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:43.859101 (+ 7696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:43.859110 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:43.859136 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:43.859719 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.859728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.861776 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.861781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.863952 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:43.863966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.866672 (+ 2706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.866678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:43.866853 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.866883 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 23:39:43.866960 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 23:39:43.867075 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12018,"mutex_wait_us":1,"thread_start_us":16721,"threads_started":1}
I20260808 23:39:43.868337 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:43.839077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.18.186.131:37047)
0808 23:39:43.858281 (+ 19204us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:43.858286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:43.858312 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:39:43.858392 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:43.858397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:43.858564 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:43.858777 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:43.860051 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.861635 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.864140 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:43.866036 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:43.866223 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:43.867809 (+ 1586us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:43.867862 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 23:39:43.867997 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19005,"thread_start_us":18819,"threads_started":1}
I20260808 23:39:43.869470 21335 tablet_service.cc:1467] Processing CreateTablet for tablet 7f43caea2d814d64a9370580663ad69e (DEFAULT_TABLE table=test-table [id=ad2653b1b59c4d5c86b3773151d19bd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:43.870108 21188 tablet_service.cc:1467] Processing CreateTablet for tablet 7f43caea2d814d64a9370580663ad69e (DEFAULT_TABLE table=test-table [id=ad2653b1b59c4d5c86b3773151d19bd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:43.871004 21335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f43caea2d814d64a9370580663ad69e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:43.871541 21188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f43caea2d814d64a9370580663ad69e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:43.871214 21487 tablet_service.cc:1467] Processing CreateTablet for tablet 7f43caea2d814d64a9370580663ad69e (DEFAULT_TABLE table=test-table [id=ad2653b1b59c4d5c86b3773151d19bd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:43.872507 21487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f43caea2d814d64a9370580663ad69e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:43.883843 21581 tablet_bootstrap.cc:492] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Bootstrap starting.
I20260808 23:39:43.885874 21582 tablet_bootstrap.cc:492] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Bootstrap starting.
I20260808 23:39:43.886916 21583 tablet_bootstrap.cc:492] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Bootstrap starting.
I20260808 23:39:43.887912 21581 tablet_bootstrap.cc:654] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:43.889238 21582 tablet_bootstrap.cc:654] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:43.889266 21581 log.cc:824] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:43.889735 21583 tablet_bootstrap.cc:654] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:43.890520 21582 log.cc:824] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:43.890998 21583 log.cc:824] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Log is configured to *not* fsync() on all Append() calls
I20260808 23:39:43.893393 21583 tablet_bootstrap.cc:492] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: No bootstrap required, opened a new log
I20260808 23:39:43.893603 21581 tablet_bootstrap.cc:492] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: No bootstrap required, opened a new log
I20260808 23:39:43.893680 21583 ts_tablet_manager.cc:1397] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260808 23:39:43.893868 21581 ts_tablet_manager.cc:1397] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260808 23:39:43.895906 21582 tablet_bootstrap.cc:492] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: No bootstrap required, opened a new log
I20260808 23:39:43.896155 21582 ts_tablet_manager.cc:1397] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260808 23:39:43.901259 21583 raft_consensus.cc:348] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:43.901662 21583 raft_consensus.cc:374] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:43.901762 21583 raft_consensus.cc:729] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02be3648e7bc4d2c969421881aa48379, State: Initialized, Role: FOLLOWER
I20260808 23:39:43.902377 21583 consensus_queue.cc:260] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:43.904227 21583 ts_tablet_manager.cc:1428] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260808 23:39:43.904945 21581 raft_consensus.cc:348] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:43.905376 21581 raft_consensus.cc:374] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:43.905483 21581 raft_consensus.cc:729] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f1edb32a3448f68d7bc17d70d67e89, State: Initialized, Role: FOLLOWER
I20260808 23:39:43.906069 21581 consensus_queue.cc:260] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:43.906669 21582 raft_consensus.cc:348] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:43.907106 21582 raft_consensus.cc:374] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:43.907205 21582 raft_consensus.cc:729] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39273e56795c47b4b9dd0fb5cbedd7bf, State: Initialized, Role: FOLLOWER
I20260808 23:39:43.907299 21561 heartbeater.cc:503] Master 127.18.186.190:44393 was elected leader, sending a full tablet report...
I20260808 23:39:43.907927 21581 ts_tablet_manager.cc:1428] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
I20260808 23:39:43.907878 21582 consensus_queue.cc:260] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:43.908951 21411 heartbeater.cc:503] Master 127.18.186.190:44393 was elected leader, sending a full tablet report...
I20260808 23:39:43.910447 21582 ts_tablet_manager.cc:1428] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
W20260808 23:39:43.963002 21263 tablet.cc:2369] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:39:43.990123 21412 tablet.cc:2369] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:39:44.008176 21562 tablet.cc:2369] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:39:44.140136 21588 raft_consensus.cc:482] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:44.140477 21588 raft_consensus.cc:504] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:44.142328 21588 leader_election.cc:288] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455), 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417)
I20260808 23:39:44.149571 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.143015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:34311 (local address 127.18.186.129:36417)
0808 23:39:44.143259 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.143268 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.143806 (+ 538us) server_negotiation.cc:407] Connection header received
0808 23:39:44.144091 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.144097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.144193 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.144279 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:44.144915 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.145840 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.147716 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.148602 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.148702 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.148752 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.149180 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.149311 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:39:44.149712 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.142738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:49269 (local address 127.18.186.130:41455)
0808 23:39:44.143005 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.143012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.143033 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:39:44.143543 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.143549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.143631 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.143736 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:44.144253 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.145489 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.147769 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.148677 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.148871 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.148952 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.149291 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.149433 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260808 23:39:44.149752 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.142836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36417 (local address 127.18.186.131:34311)
0808 23:39:44.143708 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.143726 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.143940 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.144297 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.144301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.144323 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:44.144703 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.144728 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.145979 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.145984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.147502 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.147517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.148909 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.148913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.149024 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.149047 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.149154 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.149306 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":3,"spinlock_wait_cycles":22400,"thread_start_us":478,"threads_started":1}
I20260808 23:39:44.149806 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.142559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:41455 (local address 127.18.186.131:49269)
0808 23:39:44.142912 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.142933 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.143077 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.143779 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.143785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.143827 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:44.144088 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.144100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.145715 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.145720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.147500 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.147519 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.148873 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.148879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.149113 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.149138 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.149252 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.149388 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":24,"spinlock_wait_cycles":20096}
I20260808 23:39:44.150871 21361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f43caea2d814d64a9370580663ad69e" candidate_uuid: "97f1edb32a3448f68d7bc17d70d67e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" is_pre_election: true
I20260808 23:39:44.150887 21215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f43caea2d814d64a9370580663ad69e" candidate_uuid: "97f1edb32a3448f68d7bc17d70d67e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be3648e7bc4d2c969421881aa48379" is_pre_election: true
I20260808 23:39:44.151345 21361 raft_consensus.cc:2454] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97f1edb32a3448f68d7bc17d70d67e89 in term 0.
I20260808 23:39:44.151392 21215 raft_consensus.cc:2454] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97f1edb32a3448f68d7bc17d70d67e89 in term 0.
I20260808 23:39:44.152102 21437 leader_election.cc:302] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39273e56795c47b4b9dd0fb5cbedd7bf, 97f1edb32a3448f68d7bc17d70d67e89; no voters:
I20260808 23:39:44.152498 21588 raft_consensus.cc:2781] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:44.152642 21588 raft_consensus.cc:482] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:44.152752 21588 raft_consensus.cc:3037] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.154938 21588 raft_consensus.cc:504] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:44.155567 21588 leader_election.cc:288] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [CANDIDATE]: Term 1 election: Requested vote from peers 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455), 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417)
I20260808 23:39:44.156091 21361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f43caea2d814d64a9370580663ad69e" candidate_uuid: "97f1edb32a3448f68d7bc17d70d67e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf"
I20260808 23:39:44.156091 21215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f43caea2d814d64a9370580663ad69e" candidate_uuid: "97f1edb32a3448f68d7bc17d70d67e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be3648e7bc4d2c969421881aa48379"
I20260808 23:39:44.156296 21361 raft_consensus.cc:3037] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.156296 21215 raft_consensus.cc:3037] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.159502 21361 raft_consensus.cc:2454] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97f1edb32a3448f68d7bc17d70d67e89 in term 1.
I20260808 23:39:44.159502 21215 raft_consensus.cc:2454] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97f1edb32a3448f68d7bc17d70d67e89 in term 1.
I20260808 23:39:44.159945 21434 leader_election.cc:302] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 02be3648e7bc4d2c969421881aa48379, 39273e56795c47b4b9dd0fb5cbedd7bf, 97f1edb32a3448f68d7bc17d70d67e89; no voters:
I20260808 23:39:44.160298 21588 raft_consensus.cc:2781] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:44.160740 21588 raft_consensus.cc:686] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 LEADER]: Becoming Leader. State: Replica: 97f1edb32a3448f68d7bc17d70d67e89, State: Running, Role: LEADER
I20260808 23:39:44.161193 21588 consensus_queue.cc:237] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } }
I20260808 23:39:44.165850 21041 catalog_manager.cc:5495] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131). New cstate: current_term: 1 leader_uuid: "97f1edb32a3448f68d7bc17d70d67e89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } health_report { overall_health: UNKNOWN } } }
Aug 08 23:39:44 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232382, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:44.266558 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.253862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44393 (local address 127.0.0.1:46186)
0808 23:39:44.254434 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.254454 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.254579 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.254888 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.254909 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.255636 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:44.255916 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.255925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.256873 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.256876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.257831 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.257841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.258529 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.258533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.258644 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.259325 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:44.259346 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:44.261562 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:44.264244 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:44.264257 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:44.264264 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:44.264780 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:44.265122 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:44.265125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:44.265127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:44.265248 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:44.265711 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:44.265717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:44.265869 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.266002 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.266195 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":300,"threads_started":1}
I20260808 23:39:44.266597 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.253811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46186 (local address 127.18.186.190:44393)
0808 23:39:44.253994 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.253999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.254541 (+ 542us) server_negotiation.cc:407] Connection header received
0808 23:39:44.254690 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.254694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.254755 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.254843 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:44.256055 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.256768 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.257945 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.258368 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.258465 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.261723 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:44.261727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:44.261730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:44.261758 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:44.264104 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:44.264891 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:44.264896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:44.264899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:44.265028 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:44.265331 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:44.265334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:44.265336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:44.265610 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:44.265716 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.266085 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.266319 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":50}
I20260808 23:39:44.271582 21041 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:46186:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 23:39:44.272706 21041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:44.279201 21188 tablet_service.cc:1467] Processing CreateTablet for tablet 977a7fe033a641dabdef17e8871ac9aa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=946cb6f540a94d7a96f6f95767da2f7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:39:44.279349 21335 tablet_service.cc:1467] Processing CreateTablet for tablet 977a7fe033a641dabdef17e8871ac9aa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=946cb6f540a94d7a96f6f95767da2f7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:39:44.279637 21487 tablet_service.cc:1467] Processing CreateTablet for tablet 977a7fe033a641dabdef17e8871ac9aa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=946cb6f540a94d7a96f6f95767da2f7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:39:44.280297 21188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977a7fe033a641dabdef17e8871ac9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:44.280319 21335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977a7fe033a641dabdef17e8871ac9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:44.280368 21487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977a7fe033a641dabdef17e8871ac9aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:44.285948 21582 tablet_bootstrap.cc:492] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: Bootstrap starting.
I20260808 23:39:44.287277 21582 tablet_bootstrap.cc:654] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:44.287413 21581 tablet_bootstrap.cc:492] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: Bootstrap starting.
I20260808 23:39:44.289348 21581 tablet_bootstrap.cc:654] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:44.289713 21583 tablet_bootstrap.cc:492] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: Bootstrap starting.
I20260808 23:39:44.291224 21582 tablet_bootstrap.cc:492] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: No bootstrap required, opened a new log
I20260808 23:39:44.291380 21582 ts_tablet_manager.cc:1397] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 23:39:44.291503 21583 tablet_bootstrap.cc:654] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:44.291916 21581 tablet_bootstrap.cc:492] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: No bootstrap required, opened a new log
I20260808 23:39:44.292059 21581 ts_tablet_manager.cc:1397] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 23:39:44.292254 21582 raft_consensus.cc:348] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.292506 21582 raft_consensus.cc:374] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:44.292611 21582 raft_consensus.cc:729] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39273e56795c47b4b9dd0fb5cbedd7bf, State: Initialized, Role: FOLLOWER
I20260808 23:39:44.292866 21581 raft_consensus.cc:348] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.292865 21582 consensus_queue.cc:260] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.293088 21581 raft_consensus.cc:374] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:44.293193 21581 raft_consensus.cc:729] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f1edb32a3448f68d7bc17d70d67e89, State: Initialized, Role: FOLLOWER
I20260808 23:39:44.293483 21581 consensus_queue.cc:260] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.293365 21589 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.293751 21589 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:44.293900 21588 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.294162 21588 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:44.294240 21582 ts_tablet_manager.cc:1428] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:39:44.295288 21581 ts_tablet_manager.cc:1428] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 23:39:44.295826 21583 tablet_bootstrap.cc:492] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: No bootstrap required, opened a new log
I20260808 23:39:44.295998 21583 ts_tablet_manager.cc:1397] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 23:39:44.296810 21583 raft_consensus.cc:348] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.297065 21583 raft_consensus.cc:374] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:44.297168 21583 raft_consensus.cc:729] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02be3648e7bc4d2c969421881aa48379, State: Initialized, Role: FOLLOWER
I20260808 23:39:44.297399 21583 consensus_queue.cc:260] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.297883 21587 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.298317 21587 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:44.298803 21583 ts_tablet_manager.cc:1428] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:39:44.365487 21589 raft_consensus.cc:482] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:44.365698 21589 raft_consensus.cc:504] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.367290 21589 leader_election.cc:288] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417), 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:44.374533 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.367792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:48965 (local address 127.18.186.131:37047)
0808 23:39:44.368089 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.368098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.368805 (+ 707us) server_negotiation.cc:407] Connection header received
0808 23:39:44.369075 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.369081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.369159 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.369255 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:44.369888 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.370583 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.372650 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.373520 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.373679 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.373773 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.374120 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.374246 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260808 23:39:44.374580 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.367658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:53101 (local address 127.18.186.129:36417)
0808 23:39:44.367859 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.367869 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.368635 (+ 766us) server_negotiation.cc:407] Connection header received
0808 23:39:44.368803 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.368811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.368895 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.368989 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:44.369811 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.370628 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.372691 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.373536 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.373727 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.373797 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.374167 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.374289 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:39:44.374708 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.367769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37047 (local address 127.18.186.130:48965)
0808 23:39:44.368661 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.368680 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.368831 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.369293 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.369299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.369328 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:44.369704 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.369714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.370708 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.370716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.372511 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.372523 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.373742 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.373749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.373911 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.373986 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.374061 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.374202 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":39,"thread_start_us":552,"threads_started":1}
I20260808 23:39:44.375068 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.367671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36417 (local address 127.18.186.130:53101)
0808 23:39:44.368504 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.368525 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.368659 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.369115 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.369119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.369152 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:44.369622 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.369633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.370791 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.370795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.372513 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.372529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.373791 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.373800 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.373960 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.374016 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.374122 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.374246 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":99,"spinlock_wait_cycles":24832,"thread_start_us":427,"threads_started":1}
I20260808 23:39:44.375753 21215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a7fe033a641dabdef17e8871ac9aa" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be3648e7bc4d2c969421881aa48379" is_pre_election: true
I20260808 23:39:44.375751 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a7fe033a641dabdef17e8871ac9aa" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1edb32a3448f68d7bc17d70d67e89" is_pre_election: true
I20260808 23:39:44.376035 21215 raft_consensus.cc:2454] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 0.
I20260808 23:39:44.376127 21509 raft_consensus.cc:2454] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 0.
I20260808 23:39:44.376533 21283 leader_election.cc:302] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02be3648e7bc4d2c969421881aa48379, 39273e56795c47b4b9dd0fb5cbedd7bf; no voters:
I20260808 23:39:44.376929 21589 raft_consensus.cc:2781] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:44.377061 21589 raft_consensus.cc:482] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:44.377163 21589 raft_consensus.cc:3037] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.379246 21589 raft_consensus.cc:504] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.379930 21589 leader_election.cc:288] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 election: Requested vote from peers 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417), 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:44.380461 21215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a7fe033a641dabdef17e8871ac9aa" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be3648e7bc4d2c969421881aa48379"
I20260808 23:39:44.380457 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a7fe033a641dabdef17e8871ac9aa" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1edb32a3448f68d7bc17d70d67e89"
I20260808 23:39:44.380677 21509 raft_consensus.cc:3037] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.380697 21215 raft_consensus.cc:3037] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.383698 21215 raft_consensus.cc:2454] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 1.
I20260808 23:39:44.383713 21509 raft_consensus.cc:2454] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 1.
I20260808 23:39:44.384097 21283 leader_election.cc:302] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02be3648e7bc4d2c969421881aa48379, 39273e56795c47b4b9dd0fb5cbedd7bf; no voters:
I20260808 23:39:44.384366 21589 raft_consensus.cc:2781] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:44.384810 21589 raft_consensus.cc:686] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 LEADER]: Becoming Leader. State: Replica: 39273e56795c47b4b9dd0fb5cbedd7bf, State: Running, Role: LEADER
I20260808 23:39:44.385344 21589 consensus_queue.cc:237] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.388490 21607 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: TxnStatusTablet state changed. Reason: New leader 39273e56795c47b4b9dd0fb5cbedd7bf. Latest consensus state: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.388896 21607 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:39:44.389619 21610 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 23:39:44.389892 21610 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 23:39:44.389730 21039 catalog_manager.cc:5495] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130). New cstate: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } health_report { overall_health: UNKNOWN } } }
I20260808 23:39:44.418301 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.411849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:41455 (local address 127.0.0.1:42908)
0808 23:39:44.412069 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.412089 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.412204 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.412710 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.412714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.412724 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:39:44.412901 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.412909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.414400 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.414404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.415544 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.415557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.416609 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.416620 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.416781 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.416875 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.417869 (+ 994us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.417888 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.417961 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.418076 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 23:39:44.418397 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.412018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.18.186.130:41455)
0808 23:39:44.412201 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.412212 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.412234 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:39:44.412387 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.412393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.412505 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.412667 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:44.413038 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.414248 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.415672 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.416366 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.416519 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.417067 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.417712 (+ 645us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.417864 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.417995 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.418094 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:39:44.426101 21509 raft_consensus.cc:1261] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Refusing update from remote peer 39273e56795c47b4b9dd0fb5cbedd7bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:44.426146 21215 raft_consensus.cc:1261] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Refusing update from remote peer 39273e56795c47b4b9dd0fb5cbedd7bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:44.427040 21589 consensus_queue.cc:1035] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:44.427366 21607 consensus_queue.cc:1035] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:44.434844 21588 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: TxnStatusTablet state changed. Reason: New leader 39273e56795c47b4b9dd0fb5cbedd7bf. Latest consensus state: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.435273 21588 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:44.439846 21614 mvcc.cc:205] Tried to move back new op lower bound from 7316407846599868416 to 7316407846445821952. Current Snapshot: MvccSnapshot[applied={T|T < 7316407846599868416}]
I20260808 23:39:44.440936 21589 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.441684 21613 mvcc.cc:205] Tried to move back new op lower bound from 7316407846599868416 to 7316407846445821952. Current Snapshot: MvccSnapshot[applied={T|T < 7316407846599868416}]
I20260808 23:39:44.442471 21588 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.442806 21588 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 97f1edb32a3448f68d7bc17d70d67e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:44.442800 21587 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: TxnStatusTablet state changed. Reason: New leader 39273e56795c47b4b9dd0fb5cbedd7bf. Latest consensus state: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.443094 21587 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:44.443709 21589 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:39:44.446730 21589 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.447007 21589 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 39273e56795c47b4b9dd0fb5cbedd7bf: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:39:44.449872 21587 tablet_replica.cc:439] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } } }
I20260808 23:39:44.450121 21587 tablet_replica.cc:442] T 977a7fe033a641dabdef17e8871ac9aa P 02be3648e7bc4d2c969421881aa48379: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:39:44.460079 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.454233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48104 (local address 127.18.186.131:37047)
0808 23:39:44.454404 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.454408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.454931 (+ 523us) server_negotiation.cc:407] Connection header received
0808 23:39:44.455179 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.455183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.455255 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.455413 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:44.455878 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.456640 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.457994 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.458400 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.458531 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.458954 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.459491 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.459633 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.459711 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.459794 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:39:44.476333 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.470504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:56599 (local address 127.18.186.130:41455)
0808 23:39:44.470651 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.470656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.471148 (+ 492us) server_negotiation.cc:407] Connection header received
0808 23:39:44.471383 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.471387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.471507 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.471595 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:44.472145 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.472963 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.474372 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.474883 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.475009 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.475499 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.475603 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.475700 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.475915 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.476000 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:39:44.476405 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.470395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:41455 (local address 127.18.186.131:56599)
0808 23:39:44.471037 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.471052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.471219 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.471626 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.471631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.471644 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:39:44.471992 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.472001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.473092 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.473100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.474224 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.474236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.475074 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.475085 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.475198 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.475247 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.475745 (+ 498us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.475754 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.475883 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.476017 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":381,"threads_started":1}
I20260808 23:39:44.488754 21361 raft_consensus.cc:1261] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Refusing update from remote peer 97f1edb32a3448f68d7bc17d70d67e89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:44.489195 21215 raft_consensus.cc:1261] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Refusing update from remote peer 97f1edb32a3448f68d7bc17d70d67e89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:44.489547 21591 consensus_queue.cc:1035] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:44.490022 21588 consensus_queue.cc:1035] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:44.528934 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.522245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:51347 (local address 127.18.186.131:37047)
0808 23:39:44.522464 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.522471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.523031 (+ 560us) server_negotiation.cc:407] Connection header received
0808 23:39:44.523246 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.523252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.523381 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.523522 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:44.524341 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.525340 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.526825 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.527252 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.527390 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.528135 (+ 745us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.528296 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.528392 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.528528 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.528606 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 23:39:44.529074 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.522249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37047 (local address 127.18.186.130:51347)
0808 23:39:44.522925 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.522946 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.523111 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.523537 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.523547 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.523758 (+ 211us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:39:44.524171 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.524183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.525448 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.525455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.526644 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.526656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.527582 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.527591 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.527797 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.527895 (+ 98us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.528411 (+ 516us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.528423 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.528497 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.528636 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":375,"threads_started":1}
I20260808 23:39:44.578526 21040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:44.578842 21040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:44.582930 21040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: Sending DeleteTablet for 3 replicas of tablet 7f43caea2d814d64a9370580663ad69e
I20260808 23:39:44.584033 21487 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f43caea2d814d64a9370580663ad69e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:44 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:36290
I20260808 23:39:44.584249 21188 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f43caea2d814d64a9370580663ad69e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:44 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53556
I20260808 23:39:44.585356 21335 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f43caea2d814d64a9370580663ad69e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:44 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:49876
I20260808 23:39:44.593329 21040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:39:44.594074 21040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:44.594694 21637 tablet_replica.cc:330] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: stopping tablet replica
I20260808 23:39:44.595469 21637 raft_consensus.cc:2229] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:44.601874 21187 tablet_service.cc:1467] Processing CreateTablet for tablet 1f7db646dae04694923b561699928bfc (DEFAULT_TABLE table=test-table [id=ebcd42f671af492f942ddf461b39d73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:44.602516 21187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f7db646dae04694923b561699928bfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:44.602912 21637 raft_consensus.cc:2258] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:44.602110 21335 tablet_service.cc:1467] Processing CreateTablet for tablet 1f7db646dae04694923b561699928bfc (DEFAULT_TABLE table=test-table [id=ebcd42f671af492f942ddf461b39d73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:44.603609 21335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f7db646dae04694923b561699928bfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:44.605666 21637 ts_tablet_manager.cc:1905] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:44.606985 21635 tablet_replica.cc:330] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: stopping tablet replica
I20260808 23:39:44.607605 21635 raft_consensus.cc:2229] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:39:44.609886 21583 tablet_bootstrap.cc:492] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: Bootstrap starting.
I20260808 23:39:44.608006 21486 tablet_service.cc:1467] Processing CreateTablet for tablet 1f7db646dae04694923b561699928bfc (DEFAULT_TABLE table=test-table [id=ebcd42f671af492f942ddf461b39d73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:44.610450 21486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f7db646dae04694923b561699928bfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:44.611737 21583 tablet_bootstrap.cc:654] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:44.618847 21581 tablet_bootstrap.cc:492] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: Bootstrap starting.
I20260808 23:39:44.620723 21581 tablet_bootstrap.cc:654] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:44.625994 21635 raft_consensus.cc:2258] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:44.627918 21582 tablet_bootstrap.cc:492] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: Bootstrap starting.
I20260808 23:39:44.629727 21582 tablet_bootstrap.cc:654] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:44.636029 21635 ts_tablet_manager.cc:1905] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:44.636531 21581 tablet_bootstrap.cc:492] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: No bootstrap required, opened a new log
I20260808 23:39:44.636703 21581 ts_tablet_manager.cc:1397] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.006s
I20260808 23:39:44.637562 21637 ts_tablet_manager.cc:1918] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:44.638736 21636 tablet_replica.cc:330] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: stopping tablet replica
I20260808 23:39:44.639267 21636 raft_consensus.cc:2229] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:44.639822 21581 raft_consensus.cc:348] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.640194 21581 raft_consensus.cc:374] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:44.640333 21581 raft_consensus.cc:729] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f1edb32a3448f68d7bc17d70d67e89, State: Initialized, Role: FOLLOWER
I20260808 23:39:44.640682 21581 consensus_queue.cc:260] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.641929 21581 ts_tablet_manager.cc:1428] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260808 23:39:44.638892 21582 tablet_bootstrap.cc:492] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: No bootstrap required, opened a new log
I20260808 23:39:44.643779 21582 ts_tablet_manager.cc:1397] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.005s
I20260808 23:39:44.644856 21582 raft_consensus.cc:348] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.645216 21582 raft_consensus.cc:374] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:44.645350 21582 raft_consensus.cc:729] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39273e56795c47b4b9dd0fb5cbedd7bf, State: Initialized, Role: FOLLOWER
I20260808 23:39:44.646032 21582 consensus_queue.cc:260] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.647361 21635 ts_tablet_manager.cc:1918] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:44.647528 21582 ts_tablet_manager.cc:1428] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 23:39:44.647567 21635 log.cc:1196] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/7f43caea2d814d64a9370580663ad69e
I20260808 23:39:44.647918 21637 log.cc:1196] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/7f43caea2d814d64a9370580663ad69e
I20260808 23:39:44.648164 21635 ts_tablet_manager.cc:1939] T 7f43caea2d814d64a9370580663ad69e P 97f1edb32a3448f68d7bc17d70d67e89: Deleting consensus metadata
I20260808 23:39:44.648625 21637 ts_tablet_manager.cc:1939] T 7f43caea2d814d64a9370580663ad69e P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting consensus metadata
I20260808 23:39:44.650395 21026 catalog_manager.cc:4841] TS 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455): tablet 7f43caea2d814d64a9370580663ad69e (table test-table [id=ad2653b1b59c4d5c86b3773151d19bd0]) successfully deleted
I20260808 23:39:44.640690 21583 tablet_bootstrap.cc:492] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: No bootstrap required, opened a new log
I20260808 23:39:44.651073 21583 ts_tablet_manager.cc:1397] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: Time spent bootstrapping tablet: real 0.041s user 0.007s sys 0.003s
I20260808 23:39:44.651944 21583 raft_consensus.cc:348] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.652275 21583 raft_consensus.cc:374] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:44.652396 21583 raft_consensus.cc:729] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02be3648e7bc4d2c969421881aa48379, State: Initialized, Role: FOLLOWER
I20260808 23:39:44.652738 21583 consensus_queue.cc:260] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.655692 21583 ts_tablet_manager.cc:1428] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 23:39:44.657158 21025 catalog_manager.cc:4841] TS 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047): tablet 7f43caea2d814d64a9370580663ad69e (table test-table [id=ad2653b1b59c4d5c86b3773151d19bd0]) successfully deleted
I20260808 23:39:44.658217 21636 raft_consensus.cc:2258] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:44.660666 21636 ts_tablet_manager.cc:1905] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:44.668311 21636 ts_tablet_manager.cc:1918] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:44.668555 21636 log.cc:1196] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/7f43caea2d814d64a9370580663ad69e
I20260808 23:39:44.669154 21636 ts_tablet_manager.cc:1939] T 7f43caea2d814d64a9370580663ad69e P 02be3648e7bc4d2c969421881aa48379: Deleting consensus metadata
I20260808 23:39:44.670975 21023 catalog_manager.cc:4841] TS 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417): tablet 7f43caea2d814d64a9370580663ad69e (table test-table [id=ad2653b1b59c4d5c86b3773151d19bd0]) successfully deleted
I20260808 23:39:44.794486 21587 raft_consensus.cc:482] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:44.794762 21587 raft_consensus.cc:504] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.796834 21587 leader_election.cc:288] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455), 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:44.811679 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.798191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:41407 (local address 127.18.186.130:41455)
0808 23:39:44.798612 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.798622 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.801619 (+ 2997us) server_negotiation.cc:407] Connection header received
0808 23:39:44.803264 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.803272 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.803368 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.803469 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:44.804512 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.805730 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.808770 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.810259 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.810409 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.810485 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.811159 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.811319 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260808 23:39:44.812520 21642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.797340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:41455 (local address 127.18.186.129:41407)
0808 23:39:44.801499 (+ 4159us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.801523 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.801745 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.803753 (+ 2008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.803761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.803794 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:44.804326 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.804344 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.805989 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.805995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.808289 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.808305 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.810718 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.810727 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.810932 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.810971 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.811938 (+ 967us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.812107 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4010,"thread_start_us":14116,"threads_started":1}
I20260808 23:39:44.813326 21361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f7db646dae04694923b561699928bfc" candidate_uuid: "02be3648e7bc4d2c969421881aa48379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" is_pre_election: true
I20260808 23:39:44.813663 21361 raft_consensus.cc:2454] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02be3648e7bc4d2c969421881aa48379 in term 0.
I20260808 23:39:44.814317 21137 leader_election.cc:302] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02be3648e7bc4d2c969421881aa48379, 39273e56795c47b4b9dd0fb5cbedd7bf; no voters:
I20260808 23:39:44.814792 21587 raft_consensus.cc:2781] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:44.814914 21587 raft_consensus.cc:482] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:44.814998 21587 raft_consensus.cc:3037] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.818486 21587 raft_consensus.cc:504] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.819487 21587 leader_election.cc:288] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [CANDIDATE]: Term 1 election: Requested vote from peers 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455), 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:44.820365 21361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f7db646dae04694923b561699928bfc" candidate_uuid: "02be3648e7bc4d2c969421881aa48379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf"
I20260808 23:39:44.820626 21361 raft_consensus.cc:3037] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.820830 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.798388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:58523 (local address 127.18.186.131:37047)
0808 23:39:44.798805 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.798813 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.811410 (+ 12597us) server_negotiation.cc:407] Connection header received
0808 23:39:44.812202 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.812208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.812310 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.812375 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:39:44.814105 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.815273 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.818114 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.819500 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.819630 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.819710 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.820364 (+ 654us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.820514 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260808 23:39:44.821534 21641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.797739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37047 (local address 127.18.186.129:58523)
0808 23:39:44.811304 (+ 13565us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.811325 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.811697 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.812588 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.812593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.812891 (+ 298us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:39:44.813869 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.813885 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.815495 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.815500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.817904 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.817921 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.819948 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.819955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.820153 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.820190 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.821021 (+ 831us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.821176 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13400,"thread_start_us":3504,"threads_started":1}
I20260808 23:39:44.822655 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f7db646dae04694923b561699928bfc" candidate_uuid: "02be3648e7bc4d2c969421881aa48379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1edb32a3448f68d7bc17d70d67e89" is_pre_election: true
I20260808 23:39:44.822913 21507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f7db646dae04694923b561699928bfc" candidate_uuid: "02be3648e7bc4d2c969421881aa48379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1edb32a3448f68d7bc17d70d67e89"
I20260808 23:39:44.824523 21361 raft_consensus.cc:2454] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02be3648e7bc4d2c969421881aa48379 in term 1.
I20260808 23:39:44.822942 21509 raft_consensus.cc:2454] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02be3648e7bc4d2c969421881aa48379 in term 0.
I20260808 23:39:44.825065 21137 leader_election.cc:302] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02be3648e7bc4d2c969421881aa48379, 39273e56795c47b4b9dd0fb5cbedd7bf; no voters: 97f1edb32a3448f68d7bc17d70d67e89
I20260808 23:39:44.825664 21587 raft_consensus.cc:2781] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:44.831719 21587 raft_consensus.cc:686] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 1 LEADER]: Becoming Leader. State: Replica: 02be3648e7bc4d2c969421881aa48379, State: Running, Role: LEADER
I20260808 23:39:44.832234 21587 consensus_queue.cc:237] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:44.838035 21040 catalog_manager.cc:5495] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02be3648e7bc4d2c969421881aa48379 (127.18.186.129). New cstate: current_term: 1 leader_uuid: "02be3648e7bc4d2c969421881aa48379" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } health_report { overall_health: UNKNOWN } } }
I20260808 23:39:44.895634 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.887676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54956 (local address 127.18.186.129:36417)
0808 23:39:44.888203 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.888210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.888233 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:39:44.888370 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.888377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.888502 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.889298 (+ 796us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:44.889465 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.890541 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.892623 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.893307 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.893964 (+ 657us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.894099 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.894827 (+ 728us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.895010 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.895150 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.895242 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":915,"threads_started":1}
I20260808 23:39:44.917712 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.907604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:41455 (local address 127.18.186.129:32869)
0808 23:39:44.912009 (+ 4405us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.912031 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.912179 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.912575 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.912579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.912604 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:39:44.912943 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.912956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.914482 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.914486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.915684 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.915697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.916652 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.916656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.916784 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.916843 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.917252 (+ 409us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.917261 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.917327 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.917424 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4188,"thread_start_us":4008,"threads_started":1}
I20260808 23:39:44.917717 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.907707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:32869 (local address 127.18.186.130:41455)
0808 23:39:44.907873 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.907879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.912129 (+ 4250us) server_negotiation.cc:407] Connection header received
0808 23:39:44.912348 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.912355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.912450 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.912545 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:44.913156 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.914343 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.915820 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.916467 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.916621 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.917022 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.917157 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.917256 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.917369 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.917453 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:39:44.927167 21361 raft_consensus.cc:1261] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Refusing update from remote peer 02be3648e7bc4d2c969421881aa48379: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:44.927179 21509 raft_consensus.cc:3037] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:44.927917 21643 consensus_queue.cc:1035] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:44.930576 21509 raft_consensus.cc:1261] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Refusing update from remote peer 02be3648e7bc4d2c969421881aa48379: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:44.931166 21643 consensus_queue.cc:1035] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:44.932191 21615 mvcc.cc:205] Tried to move back new op lower bound from 7316407848654254080 to 7316407848277577728. Current Snapshot: MvccSnapshot[applied={T|T < 7316407848654254080}]
I20260808 23:39:44.957468 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.952393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:41979 (local address 127.18.186.129:36417)
0808 23:39:44.952556 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:44.952563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:44.952580 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:39:44.952638 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:44.952643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:44.952739 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:44.952839 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:44.953410 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.954122 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.955607 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.956257 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.956364 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.956828 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.956918 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.956984 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:44.957137 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 23:39:44.957197 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:39:44.957547 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:44.951655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36417 (local address 127.18.186.130:41979)
0808 23:39:44.952037 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:44.952060 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:44.952179 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:44.953004 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:44.953010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:44.953022 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:39:44.953229 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.953241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.954249 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.954256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.955391 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:44.955402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:44.956431 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:44.956437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:44.956623 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:44.956675 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:39:44.957018 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:39:44.957030 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 23:39:44.957115 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 23:39:44.957259 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
W20260808 23:39:44.958814 21615 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 23:39:46.563146 21039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:46.563378 21039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:46.567632 21039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: Sending DeleteTablet for 3 replicas of tablet 1f7db646dae04694923b561699928bfc
I20260808 23:39:46.568595 21188 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f7db646dae04694923b561699928bfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53556
I20260808 23:39:46.568645 21335 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f7db646dae04694923b561699928bfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:49876
I20260808 23:39:46.568648 21486 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f7db646dae04694923b561699928bfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:36290
I20260808 23:39:46.581681 21674 tablet_replica.cc:330] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: stopping tablet replica
I20260808 23:39:46.581665 21675 tablet_replica.cc:330] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: stopping tablet replica
I20260808 23:39:46.581979 21675 raft_consensus.cc:2229] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:46.582027 21674 raft_consensus.cc:2229] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:46.582518 21674 raft_consensus.cc:2258] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:46.583082 21676 tablet_replica.cc:330] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: stopping tablet replica
I20260808 23:39:46.583346 21676 raft_consensus.cc:2229] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:39:46.583813 21674 ts_tablet_manager.cc:1905] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:46.584136 21675 raft_consensus.cc:2258] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:46.584425 21676 raft_consensus.cc:2258] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:46.585412 21676 ts_tablet_manager.cc:1905] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:46.586566 21675 ts_tablet_manager.cc:1905] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:46.591532 21674 ts_tablet_manager.cc:1918] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:39:46.591761 21674 log.cc:1196] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/1f7db646dae04694923b561699928bfc
I20260808 23:39:46.592312 21674 ts_tablet_manager.cc:1939] T 1f7db646dae04694923b561699928bfc P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting consensus metadata
I20260808 23:39:46.593425 21675 ts_tablet_manager.cc:1918] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:39:46.593762 21675 log.cc:1196] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/1f7db646dae04694923b561699928bfc
I20260808 23:39:46.593667 21026 catalog_manager.cc:4841] TS 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455): tablet 1f7db646dae04694923b561699928bfc (table test-table [id=ebcd42f671af492f942ddf461b39d73a]) successfully deleted
I20260808 23:39:46.594544 21675 ts_tablet_manager.cc:1939] T 1f7db646dae04694923b561699928bfc P 97f1edb32a3448f68d7bc17d70d67e89: Deleting consensus metadata
I20260808 23:39:46.595165 21676 ts_tablet_manager.cc:1918] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:39:46.595381 21676 log.cc:1196] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/1f7db646dae04694923b561699928bfc
I20260808 23:39:46.595719 21025 catalog_manager.cc:4841] TS 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047): tablet 1f7db646dae04694923b561699928bfc (table test-table [id=ebcd42f671af492f942ddf461b39d73a]) successfully deleted
I20260808 23:39:46.595979 21676 ts_tablet_manager.cc:1939] T 1f7db646dae04694923b561699928bfc P 02be3648e7bc4d2c969421881aa48379: Deleting consensus metadata
I20260808 23:39:46.597239 21023 catalog_manager.cc:4841] TS 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417): tablet 1f7db646dae04694923b561699928bfc (table test-table [id=ebcd42f671af492f942ddf461b39d73a]) successfully deleted
Aug 08 23:39:46 dist-test-slave-5k54 krb5kdc[20992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232386, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:39:46 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232386, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:46.636181 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:46.620901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.18.186.190:44393)
0808 23:39:46.622072 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:46.622077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:46.622096 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:39:46.622148 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:46.622151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:46.622208 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:46.622295 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:46.623679 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:46.624489 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:46.625686 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:46.626094 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:46.626196 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:46.631091 (+ 4895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:46.631096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:46.631099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:46.631133 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:46.633908 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:46.634523 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:46.634527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:46.634530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:46.634645 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:46.634937 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:46.634940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:46.634942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:46.635199 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:46.635378 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:46.635742 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 23:39:46.635916 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":909,"threads_started":1}
I20260808 23:39:46.639670 21039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:39:46.640281 21039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:46.646755 21188 tablet_service.cc:1467] Processing CreateTablet for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd (DEFAULT_TABLE table=test-table [id=59972e2d83a84ca1a4ed32b5986b8187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:46.646958 21335 tablet_service.cc:1467] Processing CreateTablet for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd (DEFAULT_TABLE table=test-table [id=59972e2d83a84ca1a4ed32b5986b8187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:46.647302 21188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:46.647207 21486 tablet_service.cc:1467] Processing CreateTablet for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd (DEFAULT_TABLE table=test-table [id=59972e2d83a84ca1a4ed32b5986b8187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:46.647430 21335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:46.647692 21486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:46.655817 21685 tablet_bootstrap.cc:492] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: Bootstrap starting.
I20260808 23:39:46.657963 21685 tablet_bootstrap.cc:654] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:46.659751 21687 tablet_bootstrap.cc:492] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: Bootstrap starting.
I20260808 23:39:46.660429 21686 tablet_bootstrap.cc:492] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: Bootstrap starting.
I20260808 23:39:46.660987 21685 tablet_bootstrap.cc:492] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: No bootstrap required, opened a new log
I20260808 23:39:46.661180 21685 ts_tablet_manager.cc:1397] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 23:39:46.662014 21685 raft_consensus.cc:348] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.662271 21686 tablet_bootstrap.cc:654] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:46.662298 21685 raft_consensus.cc:374] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:46.662402 21685 raft_consensus.cc:729] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f1edb32a3448f68d7bc17d70d67e89, State: Initialized, Role: FOLLOWER
I20260808 23:39:46.662683 21685 consensus_queue.cc:260] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.663614 21685 ts_tablet_manager.cc:1428] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:39:46.664832 21687 tablet_bootstrap.cc:654] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:46.666970 21686 tablet_bootstrap.cc:492] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: No bootstrap required, opened a new log
I20260808 23:39:46.667150 21687 tablet_bootstrap.cc:492] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: No bootstrap required, opened a new log
I20260808 23:39:46.667181 21686 ts_tablet_manager.cc:1397] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260808 23:39:46.667308 21687 ts_tablet_manager.cc:1397] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20260808 23:39:46.668093 21686 raft_consensus.cc:348] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.668314 21686 raft_consensus.cc:374] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:46.668423 21686 raft_consensus.cc:729] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02be3648e7bc4d2c969421881aa48379, State: Initialized, Role: FOLLOWER
I20260808 23:39:46.668555 21687 raft_consensus.cc:348] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.668679 21686 consensus_queue.cc:260] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.668870 21687 raft_consensus.cc:374] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:46.669054 21687 raft_consensus.cc:729] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39273e56795c47b4b9dd0fb5cbedd7bf, State: Initialized, Role: FOLLOWER
I20260808 23:39:46.669269 21686 ts_tablet_manager.cc:1428] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 23:39:46.669310 21687 consensus_queue.cc:260] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.670053 21687 ts_tablet_manager.cc:1428] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:39:46.716027 21653 raft_consensus.cc:482] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:46.716248 21653 raft_consensus.cc:504] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.717046 21653 leader_election.cc:288] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417), 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:46.717592 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cce4f9aa5f1f45bbaf60f1cd746a5ebd" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be3648e7bc4d2c969421881aa48379" is_pre_election: true
I20260808 23:39:46.717604 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cce4f9aa5f1f45bbaf60f1cd746a5ebd" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1edb32a3448f68d7bc17d70d67e89" is_pre_election: true
I20260808 23:39:46.717888 21214 raft_consensus.cc:2454] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 0.
I20260808 23:39:46.717916 21509 raft_consensus.cc:2454] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 0.
I20260808 23:39:46.718420 21283 leader_election.cc:302] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02be3648e7bc4d2c969421881aa48379, 39273e56795c47b4b9dd0fb5cbedd7bf; no voters:
I20260808 23:39:46.718734 21653 raft_consensus.cc:2781] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:46.718865 21653 raft_consensus.cc:482] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:46.719009 21653 raft_consensus.cc:3037] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:46.721426 21653 raft_consensus.cc:504] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.722004 21653 leader_election.cc:288] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 election: Requested vote from peers 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417), 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:46.722551 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cce4f9aa5f1f45bbaf60f1cd746a5ebd" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be3648e7bc4d2c969421881aa48379"
I20260808 23:39:46.722580 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cce4f9aa5f1f45bbaf60f1cd746a5ebd" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1edb32a3448f68d7bc17d70d67e89"
I20260808 23:39:46.722766 21214 raft_consensus.cc:3037] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:46.722810 21509 raft_consensus.cc:3037] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:46.725050 21214 raft_consensus.cc:2454] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 1.
I20260808 23:39:46.725462 21283 leader_election.cc:302] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02be3648e7bc4d2c969421881aa48379, 39273e56795c47b4b9dd0fb5cbedd7bf; no voters:
I20260808 23:39:46.725844 21653 raft_consensus.cc:2781] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:46.725884 21509 raft_consensus.cc:2454] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 1.
I20260808 23:39:46.726102 21653 raft_consensus.cc:686] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 LEADER]: Becoming Leader. State: Replica: 39273e56795c47b4b9dd0fb5cbedd7bf, State: Running, Role: LEADER
I20260808 23:39:46.726473 21653 consensus_queue.cc:237] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.729862 21039 catalog_manager.cc:5495] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130). New cstate: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } health_report { overall_health: UNKNOWN } } }
I20260808 23:39:46.770190 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:46.762634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.18.186.130:41455)
0808 23:39:46.763923 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:46.763928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:46.763961 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:39:46.764011 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:46.764015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:46.764076 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:46.764156 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:46.764867 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:46.765879 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:46.768097 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:46.768834 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:46.768965 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:46.769421 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:46.769529 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:46.769598 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:46.769738 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 23:39:46.769800 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1149,"thread_start_us":832,"threads_started":1}
I20260808 23:39:46.784719 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:46.779563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:38167 (local address 127.18.186.130:41455)
0808 23:39:46.779746 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:46.779750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:46.779947 (+ 197us) server_negotiation.cc:407] Connection header received
0808 23:39:46.780105 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:46.780108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:46.780166 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:46.780235 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:39:46.780738 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:46.781683 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:46.782998 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:46.783625 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:46.783716 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:46.784072 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:39:46.784162 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:39:46.784239 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:46.784353 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 23:39:46.784420 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:39:46.784861 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:46.779239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:41455 (local address 127.18.186.130:38167)
0808 23:39:46.779854 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:46.779872 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:46.780001 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:46.780277 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:46.780285 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:46.780298 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:39:46.780589 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:46.780600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:46.781806 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:46.781811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:46.782871 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:46.782883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:46.783767 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:46.783771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:46.783914 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:46.783949 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:39:46.784251 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:39:46.784258 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:39:46.784318 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 23:39:46.784412 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":317,"threads_started":1}
I20260808 23:39:46.792951 21509 raft_consensus.cc:1261] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Refusing update from remote peer 39273e56795c47b4b9dd0fb5cbedd7bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:46.793125 21214 raft_consensus.cc:1261] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Refusing update from remote peer 39273e56795c47b4b9dd0fb5cbedd7bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:46.793627 21694 consensus_queue.cc:1035] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:46.794106 21656 consensus_queue.cc:1035] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:46.839974 21040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:46.840252 21040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:46.843408 21040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: Sending DeleteTablet for 3 replicas of tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd
I20260808 23:39:46.844383 21188 tablet_service.cc:1514] Processing DeleteTablet for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53556
I20260808 23:39:46.844417 21486 tablet_service.cc:1514] Processing DeleteTablet for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:36290
I20260808 23:39:46.844661 21335 tablet_service.cc:1514] Processing DeleteTablet for tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:49876
I20260808 23:39:46.844715 21676 tablet_replica.cc:330] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: stopping tablet replica
I20260808 23:39:46.844858 21675 tablet_replica.cc:330] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: stopping tablet replica
I20260808 23:39:46.844990 21676 raft_consensus.cc:2229] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:46.845029 21674 tablet_replica.cc:330] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: stopping tablet replica
I20260808 23:39:46.845204 21676 raft_consensus.cc:2258] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:46.845201 21675 raft_consensus.cc:2229] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:46.845512 21675 raft_consensus.cc:2258] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:46.845492 21674 raft_consensus.cc:2229] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:39:46.846206 21674 raft_consensus.cc:2258] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:46.846884 21675 ts_tablet_manager.cc:1905] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:46.847640 21676 ts_tablet_manager.cc:1905] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:46.848611 21674 ts_tablet_manager.cc:1905] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:46.849243 21040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:39:46.850057 21040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:39:46.855368 21676 ts_tablet_manager.cc:1918] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:46.855508 21674 ts_tablet_manager.cc:1918] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:46.855618 21676 log.cc:1196] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/cce4f9aa5f1f45bbaf60f1cd746a5ebd
I20260808 23:39:46.855770 21674 log.cc:1196] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/cce4f9aa5f1f45bbaf60f1cd746a5ebd
I20260808 23:39:46.856223 21676 ts_tablet_manager.cc:1939] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 02be3648e7bc4d2c969421881aa48379: Deleting consensus metadata
I20260808 23:39:46.856319 21674 ts_tablet_manager.cc:1939] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting consensus metadata
I20260808 23:39:46.856592 21675 ts_tablet_manager.cc:1918] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:39:46.856763 21675 log.cc:1196] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/cce4f9aa5f1f45bbaf60f1cd746a5ebd
I20260808 23:39:46.857251 21675 ts_tablet_manager.cc:1939] T cce4f9aa5f1f45bbaf60f1cd746a5ebd P 97f1edb32a3448f68d7bc17d70d67e89: Deleting consensus metadata
I20260808 23:39:46.857535 21023 catalog_manager.cc:4841] TS 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417): tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd (table test-table [id=59972e2d83a84ca1a4ed32b5986b8187]) successfully deleted
I20260808 23:39:46.857692 21026 catalog_manager.cc:4841] TS 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455): tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd (table test-table [id=59972e2d83a84ca1a4ed32b5986b8187]) successfully deleted
I20260808 23:39:46.858325 21025 catalog_manager.cc:4841] TS 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047): tablet cce4f9aa5f1f45bbaf60f1cd746a5ebd (table test-table [id=59972e2d83a84ca1a4ed32b5986b8187]) successfully deleted
I20260808 23:39:46.862702 21188 tablet_service.cc:1467] Processing CreateTablet for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 (DEFAULT_TABLE table=test-table [id=b1a7f4e8704f47d4a783dc3118181665]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:46.862967 21335 tablet_service.cc:1467] Processing CreateTablet for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 (DEFAULT_TABLE table=test-table [id=b1a7f4e8704f47d4a783dc3118181665]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:46.863174 21188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:46.863279 21486 tablet_service.cc:1467] Processing CreateTablet for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 (DEFAULT_TABLE table=test-table [id=b1a7f4e8704f47d4a783dc3118181665]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:39:46.863448 21335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:46.863889 21486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:46.871469 21685 tablet_bootstrap.cc:492] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: Bootstrap starting.
I20260808 23:39:46.873304 21685 tablet_bootstrap.cc:654] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:46.873965 21687 tablet_bootstrap.cc:492] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: Bootstrap starting.
I20260808 23:39:46.877331 21687 tablet_bootstrap.cc:654] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:46.879544 21687 tablet_bootstrap.cc:492] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: No bootstrap required, opened a new log
I20260808 23:39:46.879722 21687 ts_tablet_manager.cc:1397] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260808 23:39:46.880388 21687 raft_consensus.cc:348] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.880611 21687 raft_consensus.cc:374] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:46.880707 21687 raft_consensus.cc:729] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39273e56795c47b4b9dd0fb5cbedd7bf, State: Initialized, Role: FOLLOWER
I20260808 23:39:46.881011 21687 consensus_queue.cc:260] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.881561 21687 ts_tablet_manager.cc:1428] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:39:46.883399 21686 tablet_bootstrap.cc:492] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: Bootstrap starting.
I20260808 23:39:46.885825 21686 tablet_bootstrap.cc:654] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:46.889195 21686 tablet_bootstrap.cc:492] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: No bootstrap required, opened a new log
I20260808 23:39:46.889495 21686 ts_tablet_manager.cc:1397] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20260808 23:39:46.889966 21685 tablet_bootstrap.cc:492] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: No bootstrap required, opened a new log
I20260808 23:39:46.890208 21685 ts_tablet_manager.cc:1397] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.013s
I20260808 23:39:46.890465 21686 raft_consensus.cc:348] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.890738 21686 raft_consensus.cc:374] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:46.890829 21686 raft_consensus.cc:729] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02be3648e7bc4d2c969421881aa48379, State: Initialized, Role: FOLLOWER
I20260808 23:39:46.890889 21685 raft_consensus.cc:348] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.891194 21685 raft_consensus.cc:374] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:46.891117 21686 consensus_queue.cc:260] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.891280 21685 raft_consensus.cc:729] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f1edb32a3448f68d7bc17d70d67e89, State: Initialized, Role: FOLLOWER
I20260808 23:39:46.891546 21685 consensus_queue.cc:260] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:46.891855 21686 ts_tablet_manager.cc:1428] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 23:39:46.892187 21685 ts_tablet_manager.cc:1428] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:39:47.217178 21694 raft_consensus.cc:482] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:39:47.217751 21694 raft_consensus.cc:504] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:47.219259 21694 leader_election.cc:288] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417), 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:47.219904 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1318d5990a1142d3bb46a7f3f1bdf7a7" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be3648e7bc4d2c969421881aa48379" is_pre_election: true
I20260808 23:39:47.220232 21214 raft_consensus.cc:2454] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 0.
I20260808 23:39:47.220711 21507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1318d5990a1142d3bb46a7f3f1bdf7a7" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1edb32a3448f68d7bc17d70d67e89" is_pre_election: true
I20260808 23:39:47.220892 21283 leader_election.cc:302] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02be3648e7bc4d2c969421881aa48379, 39273e56795c47b4b9dd0fb5cbedd7bf; no voters:
I20260808 23:39:47.221189 21507 raft_consensus.cc:2454] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 0.
I20260808 23:39:47.221342 21694 raft_consensus.cc:2781] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:39:47.221477 21694 raft_consensus.cc:482] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:39:47.221714 21694 raft_consensus.cc:3037] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:47.224293 21694 raft_consensus.cc:504] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:47.224969 21694 leader_election.cc:288] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 election: Requested vote from peers 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417), 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047)
I20260808 23:39:47.225521 21507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1318d5990a1142d3bb46a7f3f1bdf7a7" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f1edb32a3448f68d7bc17d70d67e89"
I20260808 23:39:47.225824 21507 raft_consensus.cc:3037] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:47.225868 21214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1318d5990a1142d3bb46a7f3f1bdf7a7" candidate_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02be3648e7bc4d2c969421881aa48379"
I20260808 23:39:47.226156 21214 raft_consensus.cc:3037] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:47.229313 21507 raft_consensus.cc:2454] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 1.
I20260808 23:39:47.229929 21214 raft_consensus.cc:2454] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39273e56795c47b4b9dd0fb5cbedd7bf in term 1.
I20260808 23:39:47.229904 21285 leader_election.cc:302] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39273e56795c47b4b9dd0fb5cbedd7bf, 97f1edb32a3448f68d7bc17d70d67e89; no voters:
I20260808 23:39:47.230278 21694 raft_consensus.cc:2781] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:47.230837 21694 raft_consensus.cc:686] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 LEADER]: Becoming Leader. State: Replica: 39273e56795c47b4b9dd0fb5cbedd7bf, State: Running, Role: LEADER
I20260808 23:39:47.231118 21694 consensus_queue.cc:237] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } }
I20260808 23:39:47.234532 21040 catalog_manager.cc:5495] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130). New cstate: current_term: 1 leader_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39273e56795c47b4b9dd0fb5cbedd7bf" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 41455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 } health_report { overall_health: UNKNOWN } } }
I20260808 23:39:47.271824 21214 raft_consensus.cc:1261] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Refusing update from remote peer 39273e56795c47b4b9dd0fb5cbedd7bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:47.271896 21509 raft_consensus.cc:1261] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Refusing update from remote peer 39273e56795c47b4b9dd0fb5cbedd7bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:39:47.272516 21656 consensus_queue.cc:1035] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "02be3648e7bc4d2c969421881aa48379" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:39:47.272933 21653 consensus_queue.cc:1035] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "97f1edb32a3448f68d7bc17d70d67e89" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 23:39:47.307077 21654 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 23:39:48.915830 21039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:48.916128 21039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:39:48.921033 21039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9b8f648582844b09d563f1ac1cfa096: Sending DeleteTablet for 3 replicas of tablet 1318d5990a1142d3bb46a7f3f1bdf7a7
I20260808 23:39:48.921896 21188 tablet_service.cc:1514] Processing DeleteTablet for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:48 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53556
I20260808 23:39:48.922163 21486 tablet_service.cc:1514] Processing DeleteTablet for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:48 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:36290
I20260808 23:39:48.922358 21334 tablet_service.cc:1514] Processing DeleteTablet for tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:39:48 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:49876
I20260808 23:39:48.922987 21734 tablet_replica.cc:330] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: stopping tablet replica
I20260808 23:39:48.923177 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21117
I20260808 23:39:48.923295 21734 raft_consensus.cc:2229] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:48.923581 21734 raft_consensus.cc:2258] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:48.933131 21736 tablet_replica.cc:330] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: stopping tablet replica
I20260808 23:39:48.933415 21734 ts_tablet_manager.cc:1905] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:48.933599 21736 raft_consensus.cc:2229] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:39:48.933642 21735 tablet_replica.cc:330] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: stopping tablet replica
I20260808 23:39:48.933895 21735 raft_consensus.cc:2229] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:39:48.934114 21736 raft_consensus.cc:2258] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:48.934342 21735 raft_consensus.cc:2258] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:39:48.935660 21735 ts_tablet_manager.cc:1905] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:48.935998 21736 ts_tablet_manager.cc:1905] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:39:48.943074 21736 ts_tablet_manager.cc:1918] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:39:48.943260 21736 log.cc:1196] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/1318d5990a1142d3bb46a7f3f1bdf7a7
I20260808 23:39:48.943341 21735 ts_tablet_manager.cc:1918] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:39:48.943564 21735 log.cc:1196] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/1318d5990a1142d3bb46a7f3f1bdf7a7
I20260808 23:39:48.943857 21736 ts_tablet_manager.cc:1939] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 39273e56795c47b4b9dd0fb5cbedd7bf: Deleting consensus metadata
I20260808 23:39:48.943969 21734 ts_tablet_manager.cc:1918] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:39:48.944098 21735 ts_tablet_manager.cc:1939] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 97f1edb32a3448f68d7bc17d70d67e89: Deleting consensus metadata
I20260808 23:39:48.944240 21734 log.cc:1196] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/1318d5990a1142d3bb46a7f3f1bdf7a7
I20260808 23:39:48.945010 21734 ts_tablet_manager.cc:1939] T 1318d5990a1142d3bb46a7f3f1bdf7a7 P 02be3648e7bc4d2c969421881aa48379: Deleting consensus metadata
I20260808 23:39:48.945125 21026 catalog_manager.cc:4841] TS 39273e56795c47b4b9dd0fb5cbedd7bf (127.18.186.130:41455): tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 (table test-table [id=b1a7f4e8704f47d4a783dc3118181665]) successfully deleted
I20260808 23:39:48.945178 21025 catalog_manager.cc:4841] TS 97f1edb32a3448f68d7bc17d70d67e89 (127.18.186.131:37047): tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 (table test-table [id=b1a7f4e8704f47d4a783dc3118181665]) successfully deleted
I20260808 23:39:48.946143 21023 catalog_manager.cc:4841] TS 02be3648e7bc4d2c969421881aa48379 (127.18.186.129:36417): tablet 1318d5990a1142d3bb46a7f3f1bdf7a7 (table test-table [id=b1a7f4e8704f47d4a783dc3118181665]) successfully deleted
Aug 08 23:39:48 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232386, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:39:48.964046 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:48.924395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.18.186.129:36417)
0808 23:39:48.941595 (+ 17200us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:48.941602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:48.941628 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:39:48.941690 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:48.941699 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:48.941793 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:48.943762 (+ 1969us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:48.943963 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:48.945299 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:48.946700 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:48.947104 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:48.947206 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:48.957074 (+ 9868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:48.957092 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:48.957109 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:48.957147 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:48.960967 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:48.961723 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:48.961730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:48.961733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:48.961937 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:48.962325 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:48.962328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:48.962330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:48.962740 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:48.963053 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:48.963262 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 23:39:48.963553 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17059,"thread_start_us":16971,"threads_started":1}
W20260808 23:39:48.964818 21256 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54962
W20260808 23:39:48.965477 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21117: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:48.965734 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 21117
I20260808 23:39:48.991664 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21266
Aug 08 23:39:48 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232386, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:39:49.006460 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:48.992383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42926 (local address 127.18.186.130:41455)
0808 23:39:48.992933 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:48.992938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:48.992957 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:39:48.993011 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:48.993015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:48.993081 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:48.993173 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:48.994361 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:48.995104 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:48.996411 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:48.996799 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:48.996890 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:49.001059 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:49.001073 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:49.001088 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:49.001120 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:49.003982 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:49.004645 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:49.004653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:49.004655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:49.004793 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:49.005109 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:49.005112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:49.005113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:49.005526 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:49.005748 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:49.005992 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 23:39:49.006157 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":295,"threads_started":1}
W20260808 23:39:49.007148 21406 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42926
W20260808 23:39:49.007921 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21266: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:49.008090 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 21266
I20260808 23:39:49.038842 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21415
Aug 08 23:39:49 dist-test-slave-5k54 krb5kdc[20992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232386, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:39:49.053870 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:49.039608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48116 (local address 127.18.186.131:37047)
0808 23:39:49.040163 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:49.040167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:49.040183 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:39:49.040237 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:49.040241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:49.040297 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:49.040405 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:49.041696 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:49.042447 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:49.043732 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:49.044110 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:49.044204 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:49.048404 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:49.048421 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:49.048436 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:49.048482 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:49.051348 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:49.052076 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:49.052084 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:49.052086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:49.052217 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:49.052602 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:49.052605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:49.052607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:49.052929 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:49.053199 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:49.053417 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 23:39:49.053602 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":326,"threads_started":1}
W20260808 23:39:49.054535 21556 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48116
W20260808 23:39:49.055133 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21415: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:49.055317 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 21415
I20260808 23:39:49.082831 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21006
I20260808 23:39:49.084204 21083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:49.204066 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 21006
2026-08-08T23:39:49Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8949 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:39:49 dist-test-slave-5k54 krb5kdc[21746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:39:49 dist-test-slave-5k54 krb5kdc[21746](info): set up 2 sockets
Aug 08 23:39:49 dist-test-slave-5k54 krb5kdc[21746](info): commencing operation
krb5kdc: starting...
W20260808 23:39:50.362727 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:39:50 dist-test-slave-5k54 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:39:50.570686 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:41985
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:43587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:41985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:50.894841 21759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:50.895109 21759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:50.895200 21759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:50.902267 21759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:39:50.902395 21759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:50.902478 21759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:50.902575 21759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:39:50.902657 21759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:39:50.912472 21759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:41985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:41985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:50.913765 21759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:50.915102 21759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:50.926997 21765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:50.927039 21766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:50.927057 21768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:50.927676 21759 server_base.cc:1034] running on GCE node
I20260808 23:39:50.928355 21759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:50.929593 21759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:50.930788 21759 hybrid_clock.cc:648] HybridClock initialized: now 1786232390930757 us; error 29 us; skew 500 ppm
I20260808 23:39:50.931080 21759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:50 dist-test-slave-5k54 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232390, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:50.934069 21759 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:39:50.935463 21759 webserver.cc:460] Webserver started at http://127.18.186.190:40303/ using document root <none> and password file <none>
I20260808 23:39:50.936138 21759 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:50.936266 21759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:50.936589 21759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:50.939496 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "4f9506f8b5884c459e86f2bc81c5ede3"
format_stamp: "Formatted at 2026-08-08 23:39:50 on dist-test-slave-5k54"
server_key: "64a3548edf48675db4bd02709b6e18db"
server_key_iv: "a89ef900b24dc36c2907b45b3860fdb6"
server_key_version: "encryptionkey@0"
I20260808 23:39:50.940261 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "4f9506f8b5884c459e86f2bc81c5ede3"
format_stamp: "Formatted at 2026-08-08 23:39:50 on dist-test-slave-5k54"
server_key: "64a3548edf48675db4bd02709b6e18db"
server_key_iv: "a89ef900b24dc36c2907b45b3860fdb6"
server_key_version: "encryptionkey@0"
I20260808 23:39:50.945410 21759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 23:39:50.948724 21775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:50.949992 21759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 23:39:50.950170 21759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "4f9506f8b5884c459e86f2bc81c5ede3"
format_stamp: "Formatted at 2026-08-08 23:39:50 on dist-test-slave-5k54"
server_key: "64a3548edf48675db4bd02709b6e18db"
server_key_iv: "a89ef900b24dc36c2907b45b3860fdb6"
server_key_version: "encryptionkey@0"
I20260808 23:39:50.950358 21759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:50.968116 21759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:50.972249 21759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:50.972524 21759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:50.990474 21759 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:41985
I20260808 23:39:50.990507 21826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:41985 every 8 connection(s)
I20260808 23:39:50.991851 21759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:39:50.995543 21827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:50.998759 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 21759
I20260808 23:39:50.999013 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:39:50.999840 19178 external_mini_cluster.cc:1428] Setting key 4e897ea4f5624d779e97285ab14432f1
I20260808 23:39:51.008689 21827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3: Bootstrap starting.
I20260808 23:39:51.012636 21827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:51.013998 21827 log.cc:824] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:39:51 dist-test-slave-5k54 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:51.017416 21827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3: No bootstrap required, opened a new log
I20260808 23:39:51.023721 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:51.001898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.18.186.190:41985)
0808 23:39:51.002662 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:51.002674 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:51.002716 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:39:51.004419 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:51.004444 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:51.004868 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:51.005202 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:51.007033 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:51.008432 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:51.010246 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:51.010967 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:51.011097 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:51.016514 (+ 5417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:51.016522 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:51.016536 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:51.016575 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:51.019713 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:51.020381 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:51.020391 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:51.020397 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:51.020572 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:51.020956 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:51.020963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:51.020966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:51.021478 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:51.021820 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:51.022248 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 23:39:51.022674 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":358,"threads_started":1}
I20260808 23:39:51.025660 21827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9506f8b5884c459e86f2bc81c5ede3" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41985 } }
I20260808 23:39:51.026105 21827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:51.026216 21827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9506f8b5884c459e86f2bc81c5ede3, State: Initialized, Role: FOLLOWER
I20260808 23:39:51.026861 21827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9506f8b5884c459e86f2bc81c5ede3" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41985 } }
I20260808 23:39:51.027096 21827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:39:51.027192 21827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:39:51.027315 21827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:51.029394 21827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9506f8b5884c459e86f2bc81c5ede3" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41985 } }
I20260808 23:39:51.029842 21827 leader_election.cc:302] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f9506f8b5884c459e86f2bc81c5ede3; no voters:
I20260808 23:39:51.030342 21827 leader_election.cc:288] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:39:51.030498 21832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:51.031153 21832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f9506f8b5884c459e86f2bc81c5ede3, State: Running, Role: LEADER
I20260808 23:39:51.031745 21832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9506f8b5884c459e86f2bc81c5ede3" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41985 } }
I20260808 23:39:51.032449 21827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:39:51.034720 21834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f9506f8b5884c459e86f2bc81c5ede3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9506f8b5884c459e86f2bc81c5ede3" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41985 } } }
I20260808 23:39:51.034905 21835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f9506f8b5884c459e86f2bc81c5ede3. Latest consensus state: current_term: 1 leader_uuid: "4f9506f8b5884c459e86f2bc81c5ede3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9506f8b5884c459e86f2bc81c5ede3" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41985 } } }
I20260808 23:39:51.035132 21834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:51.035239 21835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:51.037653 21843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:39:51.042753 21843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:39:51.051367 21843 catalog_manager.cc:1347] Generated new cluster ID: b01430ca8a0d42648bc177b521e80263
I20260808 23:39:51.051522 21843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:39:51.066286 21843 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:39:51.067613 21843 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:39:51.073411 21843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f9506f8b5884c459e86f2bc81c5ede3: Generated new TSK 0
I20260808 23:39:51.074450 21843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:39:51.131018 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41985
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:43587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:51.457410 21857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:51.457723 21857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:51.457892 21857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:51.465278 21857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:51.465420 21857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:51.465601 21857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:39:51.475013 21857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:51.476419 21857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:51.477691 21857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:51.490935 21863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:51.490964 21866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:51.490885 21864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:51.491775 21857 server_base.cc:1034] running on GCE node
I20260808 23:39:51.492359 21857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:51.493290 21857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:51.494565 21857 hybrid_clock.cc:648] HybridClock initialized: now 1786232391494505 us; error 60 us; skew 500 ppm
I20260808 23:39:51.494920 21857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:51 dist-test-slave-5k54 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232391, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:51.498157 21857 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:39:51.499560 21857 webserver.cc:460] Webserver started at http://127.18.186.129:32999/ using document root <none> and password file <none>
I20260808 23:39:51.500310 21857 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:51.500438 21857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:51.500746 21857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:51.503657 21857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "1f6ed42bea9f4217af583ef57053d2c6"
format_stamp: "Formatted at 2026-08-08 23:39:51 on dist-test-slave-5k54"
server_key: "a4153ba21716ff9d6e72b5d9fd3f2c85"
server_key_iv: "30844bf44fb0a298baa5975ce3aa7492"
server_key_version: "encryptionkey@0"
I20260808 23:39:51.504462 21857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "1f6ed42bea9f4217af583ef57053d2c6"
format_stamp: "Formatted at 2026-08-08 23:39:51 on dist-test-slave-5k54"
server_key: "a4153ba21716ff9d6e72b5d9fd3f2c85"
server_key_iv: "30844bf44fb0a298baa5975ce3aa7492"
server_key_version: "encryptionkey@0"
I20260808 23:39:51.509656 21857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:39:51.514312 21873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:51.515761 21857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:39:51.515962 21857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "1f6ed42bea9f4217af583ef57053d2c6"
format_stamp: "Formatted at 2026-08-08 23:39:51 on dist-test-slave-5k54"
server_key: "a4153ba21716ff9d6e72b5d9fd3f2c85"
server_key_iv: "30844bf44fb0a298baa5975ce3aa7492"
server_key_version: "encryptionkey@0"
I20260808 23:39:51.516158 21857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:51.558528 21857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:51.563079 21857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:51.563386 21857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:51.564380 21857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:51.565785 21857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:51.565861 21857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:51.566026 21857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:51.566084 21857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:51.598902 21857 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:37845
I20260808 23:39:51.598987 21990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:37845 every 8 connection(s)
I20260808 23:39:51.600246 21857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:39:51.602124 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 21857
I20260808 23:39:51.602347 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:39:51.603193 19178 external_mini_cluster.cc:1428] Setting key 8e3f11883d3cd5b744589ff3d71506af
Aug 08 23:39:51 dist-test-slave-5k54 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232391, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:51.621304 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:51.602782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:38851 (local address 127.18.186.190:41985)
0808 23:39:51.603309 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:51.603316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:51.604317 (+ 1001us) server_negotiation.cc:407] Connection header received
0808 23:39:51.606167 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:51.606176 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:51.606268 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:51.606392 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:51.607919 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:51.609081 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:51.610571 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:51.611308 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:51.611489 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:51.615415 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:51.615420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:51.615424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:51.615503 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:51.618058 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:51.618795 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:51.618800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:51.618802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:51.618928 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:51.619375 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:51.619380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:51.619383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:51.619749 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:51.619913 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:51.620690 (+ 777us) server_negotiation.cc:299] Negotiation successful
0808 23:39:51.620939 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":264,"threads_started":1}
I20260808 23:39:51.622285 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:51.603327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41985 (local address 127.18.186.129:38851)
0808 23:39:51.604167 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:51.604220 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:51.605898 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:51.606580 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:51.606587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:51.607087 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:51.607708 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:51.607726 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:51.609218 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:51.609222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:51.610429 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:51.610440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:51.611560 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:51.611565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:51.611757 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:51.612533 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:51.612568 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:51.615195 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:51.618200 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:51.618209 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:51.618224 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:51.618661 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:51.619037 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:51.619043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:51.619047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:51.619255 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:51.619879 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:51.619887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:51.620284 (+ 397us) client_negotiation.cc:769] Sending connection context
0808 23:39:51.620568 (+ 284us) client_negotiation.cc:240] Negotiation successful
0808 23:39:51.621152 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":324,"threads_started":1}
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
I20260808 23:39:51.623926 21991 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41985
I20260808 23:39:51.624251 21991 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:51.625003 21991 heartbeater.cc:511] Master 127.18.186.190:41985 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
I20260808 23:39:51.627158 21792 ts_manager.cc:194] Registered new tserver with Master: 1f6ed42bea9f4217af583ef57053d2c6 (127.18.186.129:37845)
I20260808 23:39:51.628646 21792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:38851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:39:51.651160 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41985
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:43587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:51.969571 21997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:51.969833 21997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:51.969935 21997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:51.977047 21997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:51.977151 21997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:51.977267 21997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:39:51.986646 21997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:51.987870 21997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:51.989122 21997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:52.005156 22003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:52.005558 22004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:52.006708 22006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:52.007555 21997 server_base.cc:1034] running on GCE node
I20260808 23:39:52.008150 21997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:52.009096 21997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:52.010291 21997 hybrid_clock.cc:648] HybridClock initialized: now 1786232392010249 us; error 35 us; skew 500 ppm
I20260808 23:39:52.010612 21997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:52 dist-test-slave-5k54 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232392, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:52.013816 21997 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:39:52.015173 21997 webserver.cc:460] Webserver started at http://127.18.186.130:42779/ using document root <none> and password file <none>
I20260808 23:39:52.015915 21997 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:52.016053 21997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:52.016364 21997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:52.019196 21997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "b303c9ff4fd5439eb260a47cff0f1278"
format_stamp: "Formatted at 2026-08-08 23:39:52 on dist-test-slave-5k54"
server_key: "4570bc8237cfd099f6433d2ab74e5976"
server_key_iv: "6c5cdd142e69de1a355679e78c359d31"
server_key_version: "encryptionkey@0"
I20260808 23:39:52.020107 21997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "b303c9ff4fd5439eb260a47cff0f1278"
format_stamp: "Formatted at 2026-08-08 23:39:52 on dist-test-slave-5k54"
server_key: "4570bc8237cfd099f6433d2ab74e5976"
server_key_iv: "6c5cdd142e69de1a355679e78c359d31"
server_key_version: "encryptionkey@0"
I20260808 23:39:52.025207 21997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:39:52.028496 22013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:52.029990 21997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:39:52.030156 21997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "b303c9ff4fd5439eb260a47cff0f1278"
format_stamp: "Formatted at 2026-08-08 23:39:52 on dist-test-slave-5k54"
server_key: "4570bc8237cfd099f6433d2ab74e5976"
server_key_iv: "6c5cdd142e69de1a355679e78c359d31"
server_key_version: "encryptionkey@0"
I20260808 23:39:52.030380 21997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:52.045682 21997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:52.049575 21997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:52.049868 21997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:52.050838 21997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:52.052479 21997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:52.052596 21997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:52.052801 21997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:52.052914 21997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:52.088014 21997 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:41791
I20260808 23:39:52.088061 22129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:41791 every 8 connection(s)
I20260808 23:39:52.089378 21997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:39:52.089957 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 21997
I20260808 23:39:52.090209 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:39:52.090802 19178 external_mini_cluster.cc:1428] Setting key 6f5a96a81de5fab3dc6917009d64735c
Aug 08 23:39:52 dist-test-slave-5k54 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232392, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:52.114878 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:52.092106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:37105 (local address 127.18.186.190:41985)
0808 23:39:52.092322 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:52.092328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:52.093924 (+ 1596us) server_negotiation.cc:407] Connection header received
0808 23:39:52.097246 (+ 3322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:52.097250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:52.097321 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:52.097414 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:52.099729 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:52.100531 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:52.102583 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:52.103354 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:52.103563 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:52.107563 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:52.107572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:52.107580 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:52.107631 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:52.111455 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:52.112250 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:52.112258 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:52.112265 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:52.112465 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:52.112910 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:52.112915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:52.112919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:52.113315 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:52.113485 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:52.114176 (+ 691us) server_negotiation.cc:299] Negotiation successful
0808 23:39:52.114455 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
I20260808 23:39:52.115669 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:52.092786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41985 (local address 127.18.186.130:37105)
0808 23:39:52.093720 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:52.093786 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:52.096898 (+ 3112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:52.097715 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:52.097729 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:52.098599 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:52.099545 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:52.099575 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:52.100703 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:52.100712 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:52.102403 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:52.102416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:52.103648 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:52.103654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:52.103916 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:52.104657 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:52.104680 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:52.107244 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:52.111616 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:52.111624 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:52.111639 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:52.112117 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:52.112593 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:52.112596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:52.112599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:52.112791 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:52.113441 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:52.113447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:52.113782 (+ 335us) client_negotiation.cc:769] Sending connection context
0808 23:39:52.114034 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 23:39:52.114441 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":394,"threads_started":1}
I20260808 23:39:52.117429 22130 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41985
I20260808 23:39:52.117764 22130 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
I20260808 23:39:52.118665 22130 heartbeater.cc:511] Master 127.18.186.190:41985 requested a full tablet report, sending...
I20260808 23:39:52.120457 21792 ts_manager.cc:194] Registered new tserver with Master: b303c9ff4fd5439eb260a47cff0f1278 (127.18.186.130:41791)
I20260808 23:39:52.121699 21792 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:37105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:39:52.147471 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41985
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:43587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:52.472949 22135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:52.473270 22135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:52.473444 22135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:52.480751 22135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:52.480898 22135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:52.481073 22135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:39:52.490863 22135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:52.492183 22135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:52.493453 22135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:52.506721 22140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:52.506669 22141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:52.506633 22143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:52.507684 22135 server_base.cc:1034] running on GCE node
I20260808 23:39:52.508311 22135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:52.509261 22135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:52.510480 22135 hybrid_clock.cc:648] HybridClock initialized: now 1786232392510439 us; error 40 us; skew 500 ppm
I20260808 23:39:52.510814 22135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:52 dist-test-slave-5k54 krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232392, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:52.514417 22135 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:39:52.517295 22135 webserver.cc:460] Webserver started at http://127.18.186.131:41819/ using document root <none> and password file <none>
I20260808 23:39:52.518007 22135 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:52.518139 22135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:52.518474 22135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:52.521586 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "ce76e21586bd496e90052e3e4714fa96"
format_stamp: "Formatted at 2026-08-08 23:39:52 on dist-test-slave-5k54"
server_key: "82106ee435487bf9e8cdd2d360620319"
server_key_iv: "ccaa4349af06d5ab845bf996a0a968f6"
server_key_version: "encryptionkey@0"
I20260808 23:39:52.522404 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "ce76e21586bd496e90052e3e4714fa96"
format_stamp: "Formatted at 2026-08-08 23:39:52 on dist-test-slave-5k54"
server_key: "82106ee435487bf9e8cdd2d360620319"
server_key_iv: "ccaa4349af06d5ab845bf996a0a968f6"
server_key_version: "encryptionkey@0"
I20260808 23:39:52.527796 22135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 23:39:52.531142 22150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:52.532740 22135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:39:52.532886 22135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "ce76e21586bd496e90052e3e4714fa96"
format_stamp: "Formatted at 2026-08-08 23:39:52 on dist-test-slave-5k54"
server_key: "82106ee435487bf9e8cdd2d360620319"
server_key_iv: "ccaa4349af06d5ab845bf996a0a968f6"
server_key_version: "encryptionkey@0"
I20260808 23:39:52.533051 22135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:52.549449 22135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:52.554037 22135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:52.554280 22135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:52.555318 22135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:52.556648 22135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:52.556721 22135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:52.556911 22135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:52.556967 22135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:52.589593 22135 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:39671
I20260808 23:39:52.589617 22274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:39671 every 8 connection(s)
I20260808 23:39:52.590984 22135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:39:52.598388 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 22135
I20260808 23:39:52.598626 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:39:52.599154 19178 external_mini_cluster.cc:1428] Setting key a83a44ce1f6251d3c2e7f8f94a482933
Aug 08 23:39:52 dist-test-slave-5k54 krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232392, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:52.612154 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:52.593789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:34305 (local address 127.18.186.190:41985)
0808 23:39:52.594034 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:52.594041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:52.595209 (+ 1168us) server_negotiation.cc:407] Connection header received
0808 23:39:52.597114 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:52.597118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:52.597188 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:52.597275 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:52.599069 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:52.600130 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:52.602413 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:52.602819 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:52.602926 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:52.606558 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:52.606562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:52.606566 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:52.606597 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:52.609215 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:52.609921 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:52.609930 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:52.609934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:52.610061 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:52.610504 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:52.610510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:52.610514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:52.610777 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:52.610940 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:52.611593 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 23:39:52.611808 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 23:39:52.613260 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:52.594300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41985 (local address 127.18.186.131:34305)
0808 23:39:52.595045 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:52.595083 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:52.596799 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:52.597518 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:52.597527 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:52.598060 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:52.598877 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:52.598900 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:52.600298 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:52.600311 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:52.602222 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:52.602239 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:52.603021 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:52.603030 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:52.603343 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:52.604076 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:52.604103 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:52.606347 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:52.609351 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:52.609360 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:52.609392 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:52.609794 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:52.610168 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:52.610173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:52.610177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:52.610386 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:52.610901 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:52.610908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:52.611213 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 23:39:52.611494 (+ 281us) client_negotiation.cc:240] Negotiation successful
0808 23:39:52.611965 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":268,"threads_started":1}
I20260808 23:39:52.614645 22275 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41985
I20260808 23:39:52.614945 22275 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:52.615753 22275 heartbeater.cc:511] Master 127.18.186.190:41985 requested a full tablet report, sending...
I20260808 23:39:52.617158 21791 ts_manager.cc:194] Registered new tserver with Master: ce76e21586bd496e90052e3e4714fa96 (127.18.186.131:39671)
I20260808 23:39:52.618023 21791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:34305
I20260808 23:39:52.626736 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:39:52.631369 21991 heartbeater.cc:503] Master 127.18.186.190:41985 was elected leader, sending a full tablet report...
W20260808 23:39:52.662624 21993 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:39:52.659944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.18.186.190:41985)
0808 23:39:52.660149 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:52.660159 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:52.660593 (+ 434us) server_negotiation.cc:407] Connection header received
0808 23:39:52.660817 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:52.660822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:52.660893 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:52.661005 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:52.662324 (+ 1319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50844: BlockingRecv error: recv got EOF from 127.0.0.1:50844 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:39:52.666147 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21857
W20260808 23:39:52.669281 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21857: Not authorized: Client connection negotiation failed: client connection to 127.18.186.129:37845: server requires authentication, but client does not have Kerberos credentials available
I20260808 23:39:52.669430 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 21857
I20260808 23:39:52.687644 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21997
W20260808 23:39:52.690307 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21997: Not authorized: Client connection negotiation failed: client connection to 127.18.186.130:41791: server requires authentication, but client does not have Kerberos credentials available
I20260808 23:39:52.690470 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 21997
I20260808 23:39:52.707156 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22135
W20260808 23:39:52.710042 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22135: Not authorized: Client connection negotiation failed: client connection to 127.18.186.131:39671: server requires authentication, but client does not have Kerberos credentials available
I20260808 23:39:52.710161 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 22135
I20260808 23:39:52.727890 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 21759
I20260808 23:39:52.729065 21822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:52.832676 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 21759
2026-08-08T23:39:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3628 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:39:52 dist-test-slave-5k54 krb5kdc[22293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:39:52 dist-test-slave-5k54 krb5kdc[22293](info): set up 2 sockets
Aug 08 23:39:52 dist-test-slave-5k54 krb5kdc[22293](info): commencing operation
krb5kdc: starting...
W20260808 23:39:53.963784 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.005s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:39:54 dist-test-slave-5k54 krb5kdc[22293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:39:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:39:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:39:54.198807 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:33009
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:33009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:54.524190 22307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:54.524487 22307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:54.524580 22307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:54.531921 22307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:39:54.532025 22307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:54.532079 22307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:54.532135 22307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:39:54.532183 22307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:39:54.541971 22307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:33009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:33009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:54.543253 22307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:54.544606 22307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:54.556842 22312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:54.556775 22313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:54.556902 22315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:54.557861 22307 server_base.cc:1034] running on GCE node
I20260808 23:39:54.558638 22307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:54.560101 22307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:54.561331 22307 hybrid_clock.cc:648] HybridClock initialized: now 1786232394561296 us; error 47 us; skew 500 ppm
I20260808 23:39:54.561697 22307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:54 dist-test-slave-5k54 krb5kdc[22293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232394, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:54.570395 22307 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:39:54.572105 22307 webserver.cc:460] Webserver started at http://127.18.186.190:44247/ using document root <none> and password file <none>
I20260808 23:39:54.572829 22307 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:54.572988 22307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:54.573315 22307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:54.576362 22307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "50df8364a2234b5697c287cc8fbe2977"
format_stamp: "Formatted at 2026-08-08 23:39:54 on dist-test-slave-5k54"
server_key: "210f1920507525a23bd62be12f5a9af8"
server_key_iv: "3e72a848b665d4c7ec09bee323f40d8a"
server_key_version: "encryptionkey@0"
I20260808 23:39:54.577171 22307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "50df8364a2234b5697c287cc8fbe2977"
format_stamp: "Formatted at 2026-08-08 23:39:54 on dist-test-slave-5k54"
server_key: "210f1920507525a23bd62be12f5a9af8"
server_key_iv: "3e72a848b665d4c7ec09bee323f40d8a"
server_key_version: "encryptionkey@0"
I20260808 23:39:54.582222 22307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:39:54.585518 22322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:54.586783 22307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:39:54.586968 22307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "50df8364a2234b5697c287cc8fbe2977"
format_stamp: "Formatted at 2026-08-08 23:39:54 on dist-test-slave-5k54"
server_key: "210f1920507525a23bd62be12f5a9af8"
server_key_iv: "3e72a848b665d4c7ec09bee323f40d8a"
server_key_version: "encryptionkey@0"
I20260808 23:39:54.587174 22307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:54.597774 22307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:54.601631 22307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:54.602016 22307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:54.620278 22307 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:33009
I20260808 23:39:54.620321 22377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:33009 every 8 connection(s)
I20260808 23:39:54.621659 22307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:39:54.625371 22378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:54.627828 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 22307
I20260808 23:39:54.628074 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:39:54.628741 19178 external_mini_cluster.cc:1428] Setting key 0b25330a7a5f0f8811fc01cb0570b0d2
I20260808 23:39:54.637318 22378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977: Bootstrap starting.
I20260808 23:39:54.641152 22378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:54.642462 22378 log.cc:824] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:39:54 dist-test-slave-5k54 krb5kdc[22293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:54.645419 22378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977: No bootstrap required, opened a new log
I20260808 23:39:54.652674 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:54.630681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.18.186.190:33009)
0808 23:39:54.631561 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:54.631572 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:54.631615 (+ 43us) server_negotiation.cc:407] Connection header received
0808 23:39:54.633241 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:54.633262 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:54.633649 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:54.633984 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:54.635876 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:54.637136 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:54.638879 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:54.639607 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:54.639732 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:54.644710 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:54.644718 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:54.644732 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:54.644777 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:54.647909 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:54.648628 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:54.648636 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:54.648654 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:54.648822 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:54.649223 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:54.649230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:54.649237 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:54.650067 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:54.650459 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:54.651170 (+ 711us) server_negotiation.cc:299] Negotiation successful
0808 23:39:54.651668 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":379,"threads_started":1}
I20260808 23:39:54.653919 22378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df8364a2234b5697c287cc8fbe2977" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33009 } }
I20260808 23:39:54.654291 22378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:54.654423 22378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50df8364a2234b5697c287cc8fbe2977, State: Initialized, Role: FOLLOWER
I20260808 23:39:54.654986 22378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df8364a2234b5697c287cc8fbe2977" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33009 } }
I20260808 23:39:54.655261 22378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:39:54.655406 22378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:39:54.655673 22378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:54.656747 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:39:54.658102 22378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df8364a2234b5697c287cc8fbe2977" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33009 } }
I20260808 23:39:54.658578 22378 leader_election.cc:302] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50df8364a2234b5697c287cc8fbe2977; no voters:
I20260808 23:39:54.659164 22378 leader_election.cc:288] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:39:54.659323 22383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:54.661154 22378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:39:54.667413 22383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [term 1 LEADER]: Becoming Leader. State: Replica: 50df8364a2234b5697c287cc8fbe2977, State: Running, Role: LEADER
I20260808 23:39:54.668057 22383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df8364a2234b5697c287cc8fbe2977" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33009 } }
I20260808 23:39:54.670996 22384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50df8364a2234b5697c287cc8fbe2977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df8364a2234b5697c287cc8fbe2977" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33009 } } }
I20260808 23:39:54.671412 22384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:54.671923 22385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50df8364a2234b5697c287cc8fbe2977. Latest consensus state: current_term: 1 leader_uuid: "50df8364a2234b5697c287cc8fbe2977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df8364a2234b5697c287cc8fbe2977" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33009 } } }
I20260808 23:39:54.672216 22385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977 [sys.catalog]: This master's current role is: LEADER
W20260808 23:39:54.685374 22398 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:39:54.685516 22398 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:39:54.685704 22399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:39:54.687937 22399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:39:54.697773 22399 catalog_manager.cc:1347] Generated new cluster ID: cdfd19091f4f4504a26d343f4e8598f3
I20260808 23:39:54.697909 22399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:39:54.709259 22399 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:39:54.710858 22399 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:39:54.719743 22399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50df8364a2234b5697c287cc8fbe2977: Generated new TSK 0
I20260808 23:39:54.720834 22399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:39:54.721063 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:54.710896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.18.186.190:33009)
0808 23:39:54.711096 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:54.711103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:54.711808 (+ 705us) server_negotiation.cc:407] Connection header received
0808 23:39:54.712000 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:54.712005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:54.712092 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:54.712221 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:54.713502 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:54.714605 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:54.716794 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:54.717410 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:54.717572 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:54.718787 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:54.718793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:54.718796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:39:54.719914 (+ 1118us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:54.719995 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:39:54.720043 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:54.720176 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:54.720333 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 23:39:54.720434 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 23:39:54.734900 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22307
I20260808 23:39:54.736075 22372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:54.826159 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 22307
2026-08-08T23:39:54Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1981 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:39:54 dist-test-slave-5k54 krb5kdc[22414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:39:54 dist-test-slave-5k54 krb5kdc[22414](info): set up 2 sockets
Aug 08 23:39:54 dist-test-slave-5k54 krb5kdc[22414](info): commencing operation
krb5kdc: starting...
W20260808 23:39:55.979938 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:39:56 dist-test-slave-5k54 krb5kdc[22414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:39:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:39:56.192389 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42067
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42067
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:56.530323 22427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:56.530754 22427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:56.530944 22427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:56.538311 22427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:39:56.538414 22427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:56.538516 22427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:56.538620 22427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:39:56.538686 22427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:39:56.548717 22427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:56.550052 22427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:56.551509 22427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:56.563514 22433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:56.563585 22432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:56.563565 22435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:56.564338 22427 server_base.cc:1034] running on GCE node
I20260808 23:39:56.565102 22427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:56.566398 22427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:56.567631 22427 hybrid_clock.cc:648] HybridClock initialized: now 1786232396567584 us; error 57 us; skew 500 ppm
I20260808 23:39:56.568003 22427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:56 dist-test-slave-5k54 krb5kdc[22414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232396, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:56.571117 22427 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:39:56.572640 22427 webserver.cc:460] Webserver started at http://127.18.186.190:38265/ using document root <none> and password file <none>
I20260808 23:39:56.573287 22427 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:56.573400 22427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:56.573654 22427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:56.576648 22427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "832f515183404ba7acfbf7f71ed7b7b8"
format_stamp: "Formatted at 2026-08-08 23:39:56 on dist-test-slave-5k54"
server_key: "0c95722f0c0c1ae10baaa25bf9b3d29b"
server_key_iv: "752635d4c76b9cd3ade7eac82b56c329"
server_key_version: "encryptionkey@0"
I20260808 23:39:56.577345 22427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "832f515183404ba7acfbf7f71ed7b7b8"
format_stamp: "Formatted at 2026-08-08 23:39:56 on dist-test-slave-5k54"
server_key: "0c95722f0c0c1ae10baaa25bf9b3d29b"
server_key_iv: "752635d4c76b9cd3ade7eac82b56c329"
server_key_version: "encryptionkey@0"
I20260808 23:39:56.582409 22427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:39:56.585765 22442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:56.587112 22427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:39:56.587256 22427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "832f515183404ba7acfbf7f71ed7b7b8"
format_stamp: "Formatted at 2026-08-08 23:39:56 on dist-test-slave-5k54"
server_key: "0c95722f0c0c1ae10baaa25bf9b3d29b"
server_key_iv: "752635d4c76b9cd3ade7eac82b56c329"
server_key_version: "encryptionkey@0"
I20260808 23:39:56.587507 22427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:56.621047 22427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:56.625061 22427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:56.625321 22427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:56.644641 22427 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42067
I20260808 23:39:56.644675 22495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42067 every 8 connection(s)
I20260808 23:39:56.645946 22427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:39:56.649781 22496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:39:56.652817 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 22427
I20260808 23:39:56.653039 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:39:56.653693 19178 external_mini_cluster.cc:1428] Setting key 26bf5805262630cb21808871d399f8b1
I20260808 23:39:56.663154 22496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8: Bootstrap starting.
I20260808 23:39:56.666873 22496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8: Neither blocks nor log segments found. Creating new log.
I20260808 23:39:56.667958 22496 log.cc:824] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:39:56 dist-test-slave-5k54 krb5kdc[22414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:56.671146 22496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8: No bootstrap required, opened a new log
I20260808 23:39:56.678337 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:56.655851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.18.186.190:42067)
0808 23:39:56.656649 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:56.656666 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:56.656717 (+ 51us) server_negotiation.cc:407] Connection header received
0808 23:39:56.658775 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:56.658812 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:56.659366 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:56.659877 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:56.661336 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:56.663282 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:56.664623 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:56.665697 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:56.665906 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:56.670356 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:56.670364 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:56.670378 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:56.670413 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:56.673837 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:56.674728 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:56.674739 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:56.674745 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:56.674946 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:56.675366 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:56.675369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:56.675371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:56.676048 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:56.676313 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:56.676798 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 23:39:56.677261 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":321,"threads_started":1}
I20260808 23:39:56.680331 22496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832f515183404ba7acfbf7f71ed7b7b8" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42067 } }
I20260808 23:39:56.680776 22496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:39:56.680886 22496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 832f515183404ba7acfbf7f71ed7b7b8, State: Initialized, Role: FOLLOWER
I20260808 23:39:56.681524 22496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832f515183404ba7acfbf7f71ed7b7b8" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42067 } }
I20260808 23:39:56.681782 22496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:39:56.681887 22496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:39:56.682013 22496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:39:56.684496 22496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832f515183404ba7acfbf7f71ed7b7b8" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42067 } }
I20260808 23:39:56.685060 22496 leader_election.cc:302] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 832f515183404ba7acfbf7f71ed7b7b8; no voters:
I20260808 23:39:56.685621 22496 leader_election.cc:288] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:39:56.685765 22501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:39:56.686479 22501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [term 1 LEADER]: Becoming Leader. State: Replica: 832f515183404ba7acfbf7f71ed7b7b8, State: Running, Role: LEADER
I20260808 23:39:56.687032 22501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832f515183404ba7acfbf7f71ed7b7b8" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42067 } }
I20260808 23:39:56.687810 22496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:39:56.690092 22503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "832f515183404ba7acfbf7f71ed7b7b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832f515183404ba7acfbf7f71ed7b7b8" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42067 } } }
I20260808 23:39:56.690543 22503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:56.692512 22510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:39:56.692384 22504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 832f515183404ba7acfbf7f71ed7b7b8. Latest consensus state: current_term: 1 leader_uuid: "832f515183404ba7acfbf7f71ed7b7b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832f515183404ba7acfbf7f71ed7b7b8" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42067 } } }
I20260808 23:39:56.692718 22504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8 [sys.catalog]: This master's current role is: LEADER
I20260808 23:39:56.698030 22510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:39:56.706983 22510 catalog_manager.cc:1347] Generated new cluster ID: bb1ea6294559489587c1f76fee2d2929
I20260808 23:39:56.707098 22510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:39:56.731947 22510 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:39:56.733211 22510 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:39:56.748776 22510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 832f515183404ba7acfbf7f71ed7b7b8: Generated new TSK 0
I20260808 23:39:56.749653 22510 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:39:56.811134 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42067
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:57.141894 22527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:57.142189 22527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:57.142338 22527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:57.149736 22527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:57.149860 22527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:57.150063 22527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:39:57.159767 22527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:57.161137 22527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:57.162480 22527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:57.175841 22532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:57.175745 22533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:57.175758 22536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:57.176642 22527 server_base.cc:1034] running on GCE node
I20260808 23:39:57.177295 22527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:57.178252 22527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:57.179493 22527 hybrid_clock.cc:648] HybridClock initialized: now 1786232397179448 us; error 33 us; skew 500 ppm
I20260808 23:39:57.179814 22527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:57 dist-test-slave-5k54 krb5kdc[22414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232397, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:57.183044 22527 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:39:57.184476 22527 webserver.cc:460] Webserver started at http://127.18.186.129:40077/ using document root <none> and password file <none>
I20260808 23:39:57.185154 22527 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:57.185259 22527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:57.185521 22527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:57.188582 22527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "91bc90c940844679bc8ae5e1caa02ba1"
format_stamp: "Formatted at 2026-08-08 23:39:57 on dist-test-slave-5k54"
server_key: "25cb1ebfd9ca7c422bb15f30074ccd90"
server_key_iv: "fb907e13806d30ab880ee44aa24a73a8"
server_key_version: "encryptionkey@0"
I20260808 23:39:57.189456 22527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "91bc90c940844679bc8ae5e1caa02ba1"
format_stamp: "Formatted at 2026-08-08 23:39:57 on dist-test-slave-5k54"
server_key: "25cb1ebfd9ca7c422bb15f30074ccd90"
server_key_iv: "fb907e13806d30ab880ee44aa24a73a8"
server_key_version: "encryptionkey@0"
I20260808 23:39:57.194924 22527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260808 23:39:57.198395 22543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:57.199833 22527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 23:39:57.199996 22527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "91bc90c940844679bc8ae5e1caa02ba1"
format_stamp: "Formatted at 2026-08-08 23:39:57 on dist-test-slave-5k54"
server_key: "25cb1ebfd9ca7c422bb15f30074ccd90"
server_key_iv: "fb907e13806d30ab880ee44aa24a73a8"
server_key_version: "encryptionkey@0"
I20260808 23:39:57.200187 22527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:57.212615 22527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:57.216271 22527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:57.216559 22527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:57.217450 22527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:57.218716 22527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:57.218788 22527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:57.218935 22527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:57.219022 22527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:57.253674 22527 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:39293
I20260808 23:39:57.253715 22660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:39293 every 8 connection(s)
I20260808 23:39:57.254967 22527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:39:57.263608 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 22527
I20260808 23:39:57.263862 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:39:57.264596 19178 external_mini_cluster.cc:1428] Setting key 0fe13495f3e05668019b751a2d66e7ba
Aug 08 23:39:57 dist-test-slave-5k54 krb5kdc[22414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232397, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:57.277532 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:57.257594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:44651 (local address 127.18.186.190:42067)
0808 23:39:57.258093 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:57.258100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:57.259101 (+ 1001us) server_negotiation.cc:407] Connection header received
0808 23:39:57.260893 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:57.260899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:57.260975 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:57.261082 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:57.262509 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:57.263581 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:57.265336 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:57.265879 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:57.266003 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:57.270012 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:57.270023 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:57.270031 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:57.270082 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:57.274035 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:57.274763 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:57.274770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:57.274776 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:57.274952 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:57.275398 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:57.275404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:57.275407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:57.275856 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:57.276036 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:57.276702 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 23:39:57.276986 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":230,"threads_started":1}
I20260808 23:39:57.278087 22664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:57.258226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42067 (local address 127.18.186.129:44651)
0808 23:39:57.258934 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:57.258975 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:57.260631 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:57.261274 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:57.261282 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:57.261777 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:57.262346 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:57.262363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:57.263771 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:57.263776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:57.265173 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:57.265184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:57.266122 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:57.266130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:57.266410 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:57.267123 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:57.267145 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:57.269751 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:57.274189 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:57.274195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:57.274209 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:57.274636 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:57.275087 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:57.275091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:57.275093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:57.275272 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:57.275984 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:57.275990 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:57.276299 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 23:39:57.276539 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 23:39:57.276934 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":271,"threads_started":1}
I20260808 23:39:57.279835 22661 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42067
I20260808 23:39:57.280205 22661 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:57.281210 22661 heartbeater.cc:511] Master 127.18.186.190:42067 requested a full tablet report, sending...
I20260808 23:39:57.283793 22459 ts_manager.cc:194] Registered new tserver with Master: 91bc90c940844679bc8ae5e1caa02ba1 (127.18.186.129:39293)
I20260808 23:39:57.286211 22459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:44651
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:39:57.318368 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42067
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:57.640240 22667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:57.640514 22667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:57.640605 22667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:57.647945 22667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:57.648058 22667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:57.648239 22667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:39:57.658085 22667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:57.659543 22667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:57.660915 22667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:57.673790 22675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:57.673822 22673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:57.674081 22672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:57.674916 22667 server_base.cc:1034] running on GCE node
I20260808 23:39:57.675724 22667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:57.676666 22667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:57.677856 22667 hybrid_clock.cc:648] HybridClock initialized: now 1786232397677805 us; error 51 us; skew 500 ppm
I20260808 23:39:57.678179 22667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:57 dist-test-slave-5k54 krb5kdc[22414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232397, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:57.681429 22667 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:39:57.682790 22667 webserver.cc:460] Webserver started at http://127.18.186.130:39891/ using document root <none> and password file <none>
I20260808 23:39:57.683524 22667 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:57.683660 22667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:57.683982 22667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:57.687076 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "1c4d5b7c5fdc4cb3a85ad108a7baa64c"
format_stamp: "Formatted at 2026-08-08 23:39:57 on dist-test-slave-5k54"
server_key: "ae3b55650147770e71a1d8fa896c1f17"
server_key_iv: "c554fe20dfa928d5e519a926441ed29e"
server_key_version: "encryptionkey@0"
I20260808 23:39:57.687863 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "1c4d5b7c5fdc4cb3a85ad108a7baa64c"
format_stamp: "Formatted at 2026-08-08 23:39:57 on dist-test-slave-5k54"
server_key: "ae3b55650147770e71a1d8fa896c1f17"
server_key_iv: "c554fe20dfa928d5e519a926441ed29e"
server_key_version: "encryptionkey@0"
I20260808 23:39:57.693181 22667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:39:57.696627 22682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:57.698014 22667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:39:57.698184 22667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "1c4d5b7c5fdc4cb3a85ad108a7baa64c"
format_stamp: "Formatted at 2026-08-08 23:39:57 on dist-test-slave-5k54"
server_key: "ae3b55650147770e71a1d8fa896c1f17"
server_key_iv: "c554fe20dfa928d5e519a926441ed29e"
server_key_version: "encryptionkey@0"
I20260808 23:39:57.698467 22667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:57.724296 22667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:57.728762 22667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:57.729014 22667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:57.729903 22667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:57.731128 22667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:57.731201 22667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:57.731282 22667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:57.731326 22667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:57.763867 22667 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:36591
I20260808 23:39:57.763958 22801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:36591 every 8 connection(s)
I20260808 23:39:57.765166 22667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:39:57.767961 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 22667
I20260808 23:39:57.768178 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:39:57.768688 19178 external_mini_cluster.cc:1428] Setting key 84117f4f2b6d5d245b8bf2d0a346353d
Aug 08 23:39:57 dist-test-slave-5k54 krb5kdc[22414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232397, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:57.788942 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:57.767782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:58947 (local address 127.18.186.190:42067)
0808 23:39:57.768013 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:57.768021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:57.769436 (+ 1415us) server_negotiation.cc:407] Connection header received
0808 23:39:57.771826 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:57.771830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:57.771901 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:57.771981 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:57.774106 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:57.774946 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:57.777336 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:57.777737 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:57.777832 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:57.781787 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:57.781797 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:57.781804 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:57.781853 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:57.785481 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:57.786197 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:57.786206 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:57.786212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:57.786370 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:57.786778 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:57.786785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:57.786792 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:57.787201 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:57.787394 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:57.788205 (+ 811us) server_negotiation.cc:299] Negotiation successful
0808 23:39:57.788453 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:39:57.789686 22805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:57.768407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42067 (local address 127.18.186.130:58947)
0808 23:39:57.769248 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:57.769296 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:57.771534 (+ 2238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:57.772264 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:57.772278 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:57.773112 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:57.773950 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:57.773977 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:57.775098 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:57.775107 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:57.777196 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:57.777213 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:57.777957 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:57.777967 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:57.778288 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:57.779096 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:57.779117 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:57.781550 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:57.785625 (+ 4075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:57.785632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:57.785645 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:57.786060 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:57.786474 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:57.786477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:57.786479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:57.786658 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:57.787326 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:57.787331 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:57.787718 (+ 387us) client_negotiation.cc:769] Sending connection context
0808 23:39:57.788043 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 23:39:57.788491 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":313,"threads_started":1}
I20260808 23:39:57.791193 22802 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42067
I20260808 23:39:57.791553 22802 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:57.792562 22802 heartbeater.cc:511] Master 127.18.186.190:42067 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
I20260808 23:39:57.794113 22459 ts_manager.cc:194] Registered new tserver with Master: 1c4d5b7c5fdc4cb3a85ad108a7baa64c (127.18.186.130:36591)
I20260808 23:39:57.794992 22459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:58947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:39:57.820881 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42067
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:39:58.144063 22809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:39:58.144415 22809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:39:58.144596 22809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:39:58.151964 22809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:39:58.152122 22809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:39:58.152307 22809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:39:58.162392 22809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:39:58.163838 22809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:39:58.165231 22809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:39:58.178807 22814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:58.178859 22815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:39:58.178838 22817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:39:58.179626 22809 server_base.cc:1034] running on GCE node
I20260808 23:39:58.180317 22809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:39:58.181282 22809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:39:58.182510 22809 hybrid_clock.cc:648] HybridClock initialized: now 1786232398182450 us; error 58 us; skew 500 ppm
I20260808 23:39:58.182837 22809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:39:58.186278 22809 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:39:58.187978 22809 webserver.cc:460] Webserver started at http://127.18.186.131:46159/ using document root <none> and password file <none>
I20260808 23:39:58.188736 22809 fs_manager.cc:359] Metadata directory not provided
I20260808 23:39:58.188848 22809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:39:58.189162 22809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:39:58.192426 22809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "3b03a4c1bfd64b028c2e78e11a7b5e54"
format_stamp: "Formatted at 2026-08-08 23:39:58 on dist-test-slave-5k54"
server_key: "fea10cd28f12c0117820e3fc2ce0766a"
server_key_iv: "8705ddc69930b7518013b90b3f7fffe2"
server_key_version: "encryptionkey@0"
I20260808 23:39:58.193195 22809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "3b03a4c1bfd64b028c2e78e11a7b5e54"
format_stamp: "Formatted at 2026-08-08 23:39:58 on dist-test-slave-5k54"
server_key: "fea10cd28f12c0117820e3fc2ce0766a"
server_key_iv: "8705ddc69930b7518013b90b3f7fffe2"
server_key_version: "encryptionkey@0"
I20260808 23:39:58.198828 22809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:39:58.202600 22824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:58.204362 22809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:39:58.204540 22809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "3b03a4c1bfd64b028c2e78e11a7b5e54"
format_stamp: "Formatted at 2026-08-08 23:39:58 on dist-test-slave-5k54"
server_key: "fea10cd28f12c0117820e3fc2ce0766a"
server_key_iv: "8705ddc69930b7518013b90b3f7fffe2"
server_key_version: "encryptionkey@0"
I20260808 23:39:58.204728 22809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:39:58.219806 22809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:39:58.224077 22809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:39:58.224380 22809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:39:58.225376 22809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:39:58.226765 22809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:39:58.226848 22809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:58.226989 22809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:39:58.227054 22809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:39:58.261977 22809 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:41613
I20260808 23:39:58.262017 22942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:41613 every 8 connection(s)
I20260808 23:39:58.263305 22809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:39:58.271083 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 22809
I20260808 23:39:58.271307 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:39:58.271813 19178 external_mini_cluster.cc:1428] Setting key d48b26f8a538ea3b520ac9d606ca5c40
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:58.288650 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:58.266063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:47859 (local address 127.18.186.190:42067)
0808 23:39:58.266256 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:58.266260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:58.267417 (+ 1157us) server_negotiation.cc:407] Connection header received
0808 23:39:58.269326 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:58.269331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:58.269412 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:58.269503 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:58.271330 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.272340 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.274673 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.275524 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.275670 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:58.280901 (+ 5231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:58.280910 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:58.280916 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:58.280963 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:58.284517 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.285351 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.285357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.285361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.285524 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.286023 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.286029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.286032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.286319 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:58.286494 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:58.287995 (+ 1501us) server_negotiation.cc:299] Negotiation successful
0808 23:39:58.288236 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:39:58.289800 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:58.266511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42067 (local address 127.18.186.131:47859)
0808 23:39:58.267237 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 23:39:58.267275 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:39:58.269039 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:39:58.269713 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:39:58.269721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:39:58.270250 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:39:58.271066 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:58.271094 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.272528 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.272542 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:58.274507 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:39:58.274522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.276105 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.276115 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:39:58.276548 (+ 433us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:58.277510 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:39:58.277552 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:39:58.280628 (+ 3076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:39:58.284696 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:58.284707 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:58.284734 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:58.285216 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:58.285650 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:39:58.285655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:39:58.285659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:39:58.285903 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:39:58.286523 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:39:58.286532 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:39:58.287059 (+ 527us) client_negotiation.cc:769] Sending connection context
0808 23:39:58.287879 (+ 820us) client_negotiation.cc:240] Negotiation successful
0808 23:39:58.288483 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":290,"threads_started":1}
I20260808 23:39:58.290441 22661 heartbeater.cc:503] Master 127.18.186.190:42067 was elected leader, sending a full tablet report...
I20260808 23:39:58.291980 22943 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42067
I20260808 23:39:58.292322 22943 heartbeater.cc:464] Registering TS with master...
I20260808 23:39:58.293078 22943 heartbeater.cc:511] Master 127.18.186.190:42067 requested a full tablet report, sending...
I20260808 23:39:58.294484 22459 ts_manager.cc:194] Registered new tserver with Master: 3b03a4c1bfd64b028c2e78e11a7b5e54 (127.18.186.131:41613)
I20260808 23:39:58.295369 22459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:47859
I20260808 23:39:58.300935 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232398, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232398, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:39:58.355767 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:58.340684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.18.186.190:42067)
0808 23:39:58.341025 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:58.341033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:58.341409 (+ 376us) server_negotiation.cc:407] Connection header received
0808 23:39:58.341650 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:58.341655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:58.341727 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:58.341855 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:58.343472 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.344306 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.345726 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.346248 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.346408 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:58.350485 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:58.350490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:58.350493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:58.350532 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:58.353076 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.353732 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.353739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.353743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.353881 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.354260 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.354270 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.354275 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.354616 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:58.354877 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:58.355179 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 23:39:58.355407 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
W20260808 23:39:58.356349 22459 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39104
I20260808 23:39:58.358528 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22527
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232398, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:39:58.375039 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:58.359577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45500 (local address 127.18.186.129:39293)
0808 23:39:58.360268 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:58.360275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:58.360300 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:39:58.360356 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:58.360362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:58.360574 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:58.360702 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:58.361917 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.363000 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.364426 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.365007 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.365137 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:58.369042 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:58.369053 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:58.369068 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:58.369107 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:58.372318 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.373136 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.373143 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.373146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.373270 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.373621 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.373624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.373626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.373996 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:58.374214 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:58.374476 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 23:39:58.374674 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":366,"threads_started":1}
W20260808 23:39:58.376441 22655 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45500
W20260808 23:39:58.377592 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22527: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:58.377846 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 22527
I20260808 23:39:58.396018 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22667
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232398, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:39:58.412983 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:58.396917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.18.186.130:36591)
0808 23:39:58.397571 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:58.397579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:58.397606 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:39:58.397669 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:58.397676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:58.397906 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:58.398042 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:58.399406 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.400487 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.401753 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.402347 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.402461 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:58.407102 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:58.407114 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:58.407129 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:58.407164 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:58.410414 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.411044 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.411052 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.411054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.411175 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.411576 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.411579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.411581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.411948 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:58.412182 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:58.412446 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 23:39:58.412651 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":372,"threads_started":1}
W20260808 23:39:58.414363 22797 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44614
W20260808 23:39:58.415495 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22667: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:58.415649 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 22667
I20260808 23:39:58.434194 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22809
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232398, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:39:58.451234 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:39:58.435344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47446 (local address 127.18.186.131:41613)
0808 23:39:58.436184 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0808 23:39:58.436191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:39:58.436220 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:39:58.436294 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:39:58.436301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:39:58.436524 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:39:58.436700 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:39:58.438114 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.439177 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.440552 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:39:58.441150 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:39:58.441291 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:39:58.445929 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:39:58.445941 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:39:58.445955 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:39:58.445992 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:39:58.448817 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.449456 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.449463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.449466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.449595 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:39:58.449923 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:39:58.449926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:39:58.449928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:39:58.450261 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:39:58.450496 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 23:39:58.450725 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 23:39:58.450946 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":521,"threads_started":1}
W20260808 23:39:58.452630 22938 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47446
W20260808 23:39:58.453755 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22809: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:39:58.453955 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 22809
I20260808 23:39:58.471766 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22427
I20260808 23:39:58.473038 22491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:39:58.583551 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 22427
2026-08-08T23:39:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3766 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22961](info): set up 2 sockets
Aug 08 23:39:58 dist-test-slave-5k54 krb5kdc[22961](info): commencing operation
krb5kdc: starting...
W20260808 23:39:59.711302 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:39:59 dist-test-slave-5k54 krb5kdc[22961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:39:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:39:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:39:59.922928 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:34495
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:34495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:00.250392 22974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:00.250675 22974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:00.250844 22974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:00.257966 22974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:00.258065 22974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:00.258172 22974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:00.258272 22974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:00.258358 22974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:00.267940 22974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:34495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:34495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:00.269192 22974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:00.270505 22974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:00.282181 22980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:00.282261 22979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:00.282238 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:00.283209 22974 server_base.cc:1034] running on GCE node
I20260808 23:40:00.283999 22974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:00.285351 22974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:00.286592 22974 hybrid_clock.cc:648] HybridClock initialized: now 1786232400286537 us; error 38 us; skew 500 ppm
I20260808 23:40:00.286979 22974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:00 dist-test-slave-5k54 krb5kdc[22961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232400, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:00.294342 22974 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:40:00.295881 22974 webserver.cc:460] Webserver started at http://127.18.186.190:34127/ using document root <none> and password file <none>
I20260808 23:40:00.296586 22974 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:00.296730 22974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:00.297089 22974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:00.300086 22974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "3e568848744b476bbe6f5c604f00526e"
format_stamp: "Formatted at 2026-08-08 23:40:00 on dist-test-slave-5k54"
server_key: "c960b822237117915a67d8c55a30f9ed"
server_key_iv: "8a83c9b212a59c69a223a12d61fd6fd6"
server_key_version: "encryptionkey@0"
I20260808 23:40:00.300872 22974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "3e568848744b476bbe6f5c604f00526e"
format_stamp: "Formatted at 2026-08-08 23:40:00 on dist-test-slave-5k54"
server_key: "c960b822237117915a67d8c55a30f9ed"
server_key_iv: "8a83c9b212a59c69a223a12d61fd6fd6"
server_key_version: "encryptionkey@0"
I20260808 23:40:00.305976 22974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:40:00.309268 22991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:00.310472 22974 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:40:00.310655 22974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "3e568848744b476bbe6f5c604f00526e"
format_stamp: "Formatted at 2026-08-08 23:40:00 on dist-test-slave-5k54"
server_key: "c960b822237117915a67d8c55a30f9ed"
server_key_iv: "8a83c9b212a59c69a223a12d61fd6fd6"
server_key_version: "encryptionkey@0"
I20260808 23:40:00.310849 22974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:00.327211 22974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:00.331310 22974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:00.331673 22974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:00.350606 22974 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:34495
I20260808 23:40:00.350625 23050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:34495 every 8 connection(s)
I20260808 23:40:00.351981 22974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:00.353103 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 22974
I20260808 23:40:00.353363 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:00.354015 19178 external_mini_cluster.cc:1428] Setting key e34a9208095b3dbb704df2ef701ad3c7
I20260808 23:40:00.356379 23051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:40:00 dist-test-slave-5k54 krb5kdc[22961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:00.370321 23051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e: Bootstrap starting.
I20260808 23:40:00.374517 23051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:00.375780 23051 log.cc:824] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:00.379000 23051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e: No bootstrap required, opened a new log
I20260808 23:40:00.379244 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:00.356007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.18.186.190:34495)
0808 23:40:00.356789 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:00.356803 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:00.356849 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:40:00.358496 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:00.358517 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:00.359376 (+ 859us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:00.359880 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:00.362196 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:00.363657 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:00.365390 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:00.366053 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:00.366189 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:00.370869 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:00.370879 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:00.370897 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:00.370940 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:00.374806 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:00.375493 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:00.375504 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:00.375514 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:00.375732 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:00.376150 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:00.376157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:00.376160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:00.376796 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:00.377006 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:00.377537 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 23:40:00.377939 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":367,"threads_started":1}
I20260808 23:40:00.387181 23051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e568848744b476bbe6f5c604f00526e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34495 } }
I20260808 23:40:00.387555 23051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:00.387637 23051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e568848744b476bbe6f5c604f00526e, State: Initialized, Role: FOLLOWER
I20260808 23:40:00.388108 23051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e568848744b476bbe6f5c604f00526e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34495 } }
I20260808 23:40:00.388304 23051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:00.388397 23051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:00.388522 23051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:00.390512 23051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e568848744b476bbe6f5c604f00526e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34495 } }
I20260808 23:40:00.391052 23051 leader_election.cc:302] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e568848744b476bbe6f5c604f00526e; no voters:
I20260808 23:40:00.391767 23051 leader_election.cc:288] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:00.392051 23056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:00.392746 23056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [term 1 LEADER]: Becoming Leader. State: Replica: 3e568848744b476bbe6f5c604f00526e, State: Running, Role: LEADER
I20260808 23:40:00.393262 23056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e568848744b476bbe6f5c604f00526e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34495 } }
I20260808 23:40:00.393966 23051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:00.396351 23058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e568848744b476bbe6f5c604f00526e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e568848744b476bbe6f5c604f00526e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34495 } } }
I20260808 23:40:00.396273 23059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e568848744b476bbe6f5c604f00526e. Latest consensus state: current_term: 1 leader_uuid: "3e568848744b476bbe6f5c604f00526e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e568848744b476bbe6f5c604f00526e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34495 } } }
I20260808 23:40:00.396790 23058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:00.396801 23059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:00.399679 23067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:00.404585 23067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:00.412823 23067 catalog_manager.cc:1347] Generated new cluster ID: e8386af5e4864c25a514dfe2b817d83c
I20260808 23:40:00.412961 23067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:00.440268 23067 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:00.441668 23067 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:00.449851 23067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e568848744b476bbe6f5c604f00526e: Generated new TSK 0
I20260808 23:40:00.450739 23067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:40:00.502923 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34495
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:00.831240 23079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:00.831537 23079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:00.831738 23079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:00.838809 23079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:00.838907 23079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:00.839118 23079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:00.849042 23079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:00.850291 23079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:00.851742 23079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:00.864845 23085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:00.865339 23087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:00.865449 23089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:00.866140 23079 server_base.cc:1034] running on GCE node
I20260808 23:40:00.866744 23079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:00.867759 23079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:00.868961 23079 hybrid_clock.cc:648] HybridClock initialized: now 1786232400868924 us; error 38 us; skew 500 ppm
I20260808 23:40:00.869313 23079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:00 dist-test-slave-5k54 krb5kdc[22961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232400, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:00.872594 23079 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:40:00.874603 23079 webserver.cc:460] Webserver started at http://127.18.186.129:42531/ using document root <none> and password file <none>
I20260808 23:40:00.875545 23079 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:00.875689 23079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:00.876077 23079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:00.880059 23079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "d7a73a1bc4f44eb982a6d0615619b7a8"
format_stamp: "Formatted at 2026-08-08 23:40:00 on dist-test-slave-5k54"
server_key: "4e8cf361ea5193121af724f1ddbbbe34"
server_key_iv: "5a9c14091af64b0b8e704281ce7fbdf0"
server_key_version: "encryptionkey@0"
I20260808 23:40:00.880870 23079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "d7a73a1bc4f44eb982a6d0615619b7a8"
format_stamp: "Formatted at 2026-08-08 23:40:00 on dist-test-slave-5k54"
server_key: "4e8cf361ea5193121af724f1ddbbbe34"
server_key_iv: "5a9c14091af64b0b8e704281ce7fbdf0"
server_key_version: "encryptionkey@0"
I20260808 23:40:00.886303 23079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:40:00.889811 23096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:00.891160 23079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 23:40:00.891335 23079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "d7a73a1bc4f44eb982a6d0615619b7a8"
format_stamp: "Formatted at 2026-08-08 23:40:00 on dist-test-slave-5k54"
server_key: "4e8cf361ea5193121af724f1ddbbbe34"
server_key_iv: "5a9c14091af64b0b8e704281ce7fbdf0"
server_key_version: "encryptionkey@0"
I20260808 23:40:00.891602 23079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:00.910280 23079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:00.913993 23079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:00.914333 23079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:00.915325 23079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:00.916676 23079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:00.916778 23079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:00.916927 23079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:00.916998 23079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:00.950155 23079 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:44597
I20260808 23:40:00.950191 23220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:44597 every 8 connection(s)
I20260808 23:40:00.951547 23079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:00.954344 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 23079
I20260808 23:40:00.954609 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:40:00.955273 19178 external_mini_cluster.cc:1428] Setting key 64a6d94bc07bb93830dd0edbf791941e
Aug 08 23:40:00 dist-test-slave-5k54 krb5kdc[22961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232400, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:00.974768 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:00.954208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:47209 (local address 127.18.186.190:34495)
0808 23:40:00.954701 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:00.954708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:00.955879 (+ 1171us) server_negotiation.cc:407] Connection header received
0808 23:40:00.958214 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:00.958219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:00.958277 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:00.958360 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:00.960474 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:00.961171 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:00.963641 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:00.964041 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:00.964136 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:00.967887 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:00.967896 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:00.967911 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:00.967956 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:00.971512 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:00.972229 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:00.972238 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:00.972244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:00.972418 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:00.972833 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:00.972841 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:00.972847 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:00.973258 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:00.973411 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:00.974058 (+ 647us) server_negotiation.cc:299] Negotiation successful
0808 23:40:00.974302 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":241,"threads_started":1}
I20260808 23:40:00.975520 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:00.954816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34495 (local address 127.18.186.129:47209)
0808 23:40:00.955688 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:00.955745 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:00.957923 (+ 2178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:00.958641 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:00.958655 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:00.959516 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:00.960307 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:00.960335 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:00.961338 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:00.961346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:00.963508 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:00.963526 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:00.964271 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:00.964280 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:00.964622 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:00.965422 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:00.965448 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:00.967653 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:00.971652 (+ 3999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:00.971659 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:00.971670 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:00.972093 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:00.972534 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:00.972537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:00.972539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:00.972707 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:00.973372 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:00.973377 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:00.973679 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 23:40:00.973893 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 23:40:00.974251 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":344,"threads_started":1}
I20260808 23:40:00.977157 23221 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34495
I20260808 23:40:00.977499 23221 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:00.978489 23221 heartbeater.cc:511] Master 127.18.186.190:34495 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
I20260808 23:40:00.980885 23011 ts_manager.cc:194] Registered new tserver with Master: d7a73a1bc4f44eb982a6d0615619b7a8 (127.18.186.129:44597)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
I20260808 23:40:00.982600 23011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:47209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:40:01.007228 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34495
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:01.329087 23227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:01.329357 23227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:01.329457 23227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:01.336685 23227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:01.336787 23227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:01.336899 23227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:01.346927 23227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:01.348206 23227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:01.349485 23227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:01.364109 23233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:01.364428 23236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:01.364105 23234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:01.365617 23227 server_base.cc:1034] running on GCE node
I20260808 23:40:01.366425 23227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:01.367363 23227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:01.368626 23227 hybrid_clock.cc:648] HybridClock initialized: now 1786232401368571 us; error 47 us; skew 500 ppm
I20260808 23:40:01.368942 23227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:01 dist-test-slave-5k54 krb5kdc[22961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232401, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:01.372195 23227 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:40:01.373744 23227 webserver.cc:460] Webserver started at http://127.18.186.130:40607/ using document root <none> and password file <none>
I20260808 23:40:01.374449 23227 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:01.374595 23227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:01.374907 23227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:01.377895 23227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "66f5082a64ab438fb3f4ee7883f82d40"
format_stamp: "Formatted at 2026-08-08 23:40:01 on dist-test-slave-5k54"
server_key: "0018b6303da85de691a2bea02bd7e5a5"
server_key_iv: "34de090ee0098124b5d2da768eb0b833"
server_key_version: "encryptionkey@0"
I20260808 23:40:01.378710 23227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "66f5082a64ab438fb3f4ee7883f82d40"
format_stamp: "Formatted at 2026-08-08 23:40:01 on dist-test-slave-5k54"
server_key: "0018b6303da85de691a2bea02bd7e5a5"
server_key_iv: "34de090ee0098124b5d2da768eb0b833"
server_key_version: "encryptionkey@0"
I20260808 23:40:01.383930 23227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 23:40:01.387277 23244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:01.388557 23227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:40:01.388731 23227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "66f5082a64ab438fb3f4ee7883f82d40"
format_stamp: "Formatted at 2026-08-08 23:40:01 on dist-test-slave-5k54"
server_key: "0018b6303da85de691a2bea02bd7e5a5"
server_key_iv: "34de090ee0098124b5d2da768eb0b833"
server_key_version: "encryptionkey@0"
I20260808 23:40:01.388939 23227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:01.412031 23227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:01.416678 23227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:01.417042 23227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:01.418026 23227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:01.419333 23227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:01.419456 23227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:01.419595 23227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:01.419680 23227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:01.453405 23227 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:37963
I20260808 23:40:01.453418 23368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:37963 every 8 connection(s)
I20260808 23:40:01.454762 23227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:01.457934 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 23227
I20260808 23:40:01.458125 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:40:01.458566 19178 external_mini_cluster.cc:1428] Setting key 2a329c1a178277ccbb88948a01fdcf8f
Aug 08 23:40:01 dist-test-slave-5k54 krb5kdc[22961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232401, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:01.478531 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:01.457416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:33231 (local address 127.18.186.190:34495)
0808 23:40:01.457634 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:01.457642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:01.459067 (+ 1425us) server_negotiation.cc:407] Connection header received
0808 23:40:01.461277 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:01.461281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:01.461345 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:01.461424 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:01.463624 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:01.464305 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:01.466835 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:01.467280 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:01.467376 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:01.471253 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:01.471262 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:01.471270 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:01.471317 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:01.475020 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:01.475871 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:01.475879 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:01.475886 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:01.476077 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:01.476569 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:01.476577 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:01.476583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:01.476963 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:01.477125 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:01.477804 (+ 679us) server_negotiation.cc:299] Negotiation successful
0808 23:40:01.478066 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:40:01.479231 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:01.457937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34495 (local address 127.18.186.130:33231)
0808 23:40:01.458862 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:01.458929 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:01.460967 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:01.461721 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:01.461735 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:01.462581 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:01.463449 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:01.463481 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:01.464478 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:01.464487 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:01.466678 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:01.466697 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:01.467521 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:01.467532 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:01.467857 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:01.468589 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:01.468623 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:01.471024 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:01.475165 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:01.475173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:01.475206 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:01.475736 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:01.476216 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:01.476223 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:01.476230 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:01.476445 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:01.477093 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:01.477099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:01.477415 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 23:40:01.477643 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 23:40:01.478026 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":412,"threads_started":1}
I20260808 23:40:01.480756 23369 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34495
I20260808 23:40:01.481102 23369 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:01.482016 23369 heartbeater.cc:511] Master 127.18.186.190:34495 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
I20260808 23:40:01.483757 23011 ts_manager.cc:194] Registered new tserver with Master: 66f5082a64ab438fb3f4ee7883f82d40 (127.18.186.130:37963)
I20260808 23:40:01.484622 23011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:33231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:40:01.514662 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34495
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:01.843029 23374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:01.843338 23374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:01.843547 23374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:01.850775 23374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:01.850943 23374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:01.851117 23374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:01.860893 23374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:01.862164 23374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:01.863509 23374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:01.876542 23379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:01.876461 23382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:01.876484 23380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:01.877455 23374 server_base.cc:1034] running on GCE node
I20260808 23:40:01.878036 23374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:01.878996 23374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:01.880241 23374 hybrid_clock.cc:648] HybridClock initialized: now 1786232401880188 us; error 68 us; skew 500 ppm
I20260808 23:40:01.880579 23374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:01 dist-test-slave-5k54 krb5kdc[22961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232401, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:01.883896 23374 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:40:01.885272 23374 webserver.cc:460] Webserver started at http://127.18.186.131:44379/ using document root <none> and password file <none>
I20260808 23:40:01.885926 23374 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:01.886029 23374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:01.886283 23374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:01.889168 23374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "a591fe341dc84456877b58ea1c5ae71a"
format_stamp: "Formatted at 2026-08-08 23:40:01 on dist-test-slave-5k54"
server_key: "a5fd4060d4e57c42e1a3e6b49b912b59"
server_key_iv: "b168ba334c95f781f2c2dc51797a3aae"
server_key_version: "encryptionkey@0"
I20260808 23:40:01.889896 23374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "a591fe341dc84456877b58ea1c5ae71a"
format_stamp: "Formatted at 2026-08-08 23:40:01 on dist-test-slave-5k54"
server_key: "a5fd4060d4e57c42e1a3e6b49b912b59"
server_key_iv: "b168ba334c95f781f2c2dc51797a3aae"
server_key_version: "encryptionkey@0"
I20260808 23:40:01.895005 23374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 23:40:01.898722 23389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:01.900095 23374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:01.900267 23374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "a591fe341dc84456877b58ea1c5ae71a"
format_stamp: "Formatted at 2026-08-08 23:40:01 on dist-test-slave-5k54"
server_key: "a5fd4060d4e57c42e1a3e6b49b912b59"
server_key_iv: "b168ba334c95f781f2c2dc51797a3aae"
server_key_version: "encryptionkey@0"
I20260808 23:40:01.900465 23374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:01.931941 23374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:01.936339 23374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:01.936648 23374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:01.937651 23374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:01.939034 23374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:01.939137 23374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:01.939282 23374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:01.939363 23374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:01.974615 23374 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:44783
I20260808 23:40:01.974639 23509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:44783 every 8 connection(s)
I20260808 23:40:01.976030 23374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:01.976531 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 23374
I20260808 23:40:01.976747 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:40:01.977377 19178 external_mini_cluster.cc:1428] Setting key 8fd76a4afecf5668cb89cc9eb1bb0173
I20260808 23:40:01.985483 23221 heartbeater.cc:503] Master 127.18.186.190:34495 was elected leader, sending a full tablet report...
Aug 08 23:40:01 dist-test-slave-5k54 krb5kdc[22961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232401, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:01.998248 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:01.978980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:53703 (local address 127.18.186.190:34495)
0808 23:40:01.979183 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:01.979188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:01.981002 (+ 1814us) server_negotiation.cc:407] Connection header received
0808 23:40:01.982875 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:01.982880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:01.982976 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:01.983064 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:01.984684 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:01.985630 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:01.987691 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:01.988108 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:01.988207 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:01.991816 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:01.991824 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:01.991828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:01.991868 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:01.995358 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:01.996077 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:01.996082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:01.996086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:01.996217 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:01.996684 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:01.996687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:01.996689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:01.996982 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:01.997128 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:01.997822 (+ 694us) server_negotiation.cc:299] Negotiation successful
0808 23:40:01.997983 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:40:01.999161 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:01.979855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34495 (local address 127.18.186.131:53703)
0808 23:40:01.980790 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:01.980870 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:01.982619 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:01.983357 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:01.983370 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:01.983948 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:01.984511 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:01.984528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:01.985773 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:01.985777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:01.987568 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:01.987580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:01.988293 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:01.988298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:01.988491 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:01.989129 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:01.989150 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:01.991537 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:01.995524 (+ 3987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:01.995532 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:01.995548 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:01.995951 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:01.996362 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:01.996373 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:01.996376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:01.996566 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:01.997087 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:01.997092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:01.997441 (+ 349us) client_negotiation.cc:769] Sending connection context
0808 23:40:01.997660 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 23:40:01.998030 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":381,"threads_started":1}
I20260808 23:40:02.000633 23510 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34495
I20260808 23:40:02.000939 23510 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:02.001637 23510 heartbeater.cc:511] Master 127.18.186.190:34495 requested a full tablet report, sending...
I20260808 23:40:02.003031 23011 ts_manager.cc:194] Registered new tserver with Master: a591fe341dc84456877b58ea1c5ae71a (127.18.186.131:44783)
I20260808 23:40:02.004182 23011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:53703
I20260808 23:40:02.006103 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:40:02 dist-test-slave-5k54 krb5kdc[22961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:40:02 dist-test-slave-5k54 krb5kdc[22961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:40:02.059406 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:02.043935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.18.186.129:44597)
0808 23:40:02.044591 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:02.044598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:02.044626 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:40:02.044688 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:02.044694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:02.044942 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:02.045078 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:02.046519 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.047628 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.048842 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.049432 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.049547 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:02.053677 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:02.053692 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:02.053706 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:02.053743 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:02.056801 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.057428 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.057436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.057438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.057561 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.057916 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.057920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.057922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.058282 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:02.058524 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:02.058787 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 23:40:02.058986 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":342,"threads_started":1}
Aug 08 23:40:02 dist-test-slave-5k54 krb5kdc[22961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:02.077229 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:02.063982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46024 (local address 127.18.186.190:34495)
0808 23:40:02.064213 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:02.064219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:02.064322 (+ 103us) server_negotiation.cc:407] Connection header received
0808 23:40:02.064428 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:02.064432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:02.064493 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:02.064575 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:02.065812 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.066529 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.067758 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.068150 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.068250 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:02.072107 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:02.072113 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:02.072116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:02.072158 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:02.074710 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.075323 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.075326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.075328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.075617 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.075951 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.075954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.075956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.076276 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:02.076470 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:02.076716 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 23:40:02.076885 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260808 23:40:02.077786 23010 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46024
I20260808 23:40:02.078809 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 23079
I20260808 23:40:02.091336 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:02.079770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.18.186.129:44597)
0808 23:40:02.080002 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:02.080009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:02.080169 (+ 160us) server_negotiation.cc:407] Connection header received
0808 23:40:02.080359 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:02.080363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:02.080454 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:02.080537 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:02.081675 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.082344 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.083633 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.084056 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.084203 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:02.086357 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:02.086361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:02.086365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:02.086395 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:02.088844 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.089491 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.089497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.089501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.089627 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.090005 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.090011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.090014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.090259 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:02.090515 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:02.090801 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 23:40:02.091023 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:40:02.092026 23214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:02.174710 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 23079
I20260808 23:40:02.195739 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 23227
Aug 08 23:40:02 dist-test-slave-5k54 krb5kdc[22961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:40:02.212371 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:02.196786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.18.186.130:37963)
0808 23:40:02.197432 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:02.197439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:02.197466 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:40:02.197529 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:02.197535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:02.197795 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:02.197934 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:02.199337 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.200566 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.201840 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.202407 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.202508 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:02.206766 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:02.206778 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:02.206792 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:02.206834 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:02.209829 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.210440 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.210446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.210449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.210575 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.210892 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.210895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.210897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.211228 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:02.211750 (+ 522us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:02.211873 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 23:40:02.212085 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":346,"threads_started":1}
I20260808 23:40:02.213636 23364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:02.300354 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 23227
I20260808 23:40:02.321349 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 23374
Aug 08 23:40:02 dist-test-slave-5k54 krb5kdc[22961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:40:02.347640 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:02.322901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.18.186.131:44783)
0808 23:40:02.324075 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:02.324091 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:02.324137 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:40:02.324230 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:02.324244 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:02.324575 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:02.324782 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:02.326841 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.328744 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.330956 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:02.332033 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:02.332174 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:02.339065 (+ 6891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:02.339079 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:02.339103 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:02.339156 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:02.343968 (+ 4812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.345034 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.345050 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.345053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.345239 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:02.345561 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:02.345564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:02.345566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:02.345881 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:02.346107 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:02.346539 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 23:40:02.346869 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":653,"threads_started":1}
I20260808 23:40:02.349660 23504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:02.467135 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 23374
I20260808 23:40:02.494345 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 22974
I20260808 23:40:02.495642 23046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:02.637178 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 22974
2026-08-08T23:40:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4069 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:40:02 dist-test-slave-5k54 krb5kdc[23532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:40:02 dist-test-slave-5k54 krb5kdc[23532](info): set up 2 sockets
Aug 08 23:40:02 dist-test-slave-5k54 krb5kdc[23532](info): commencing operation
krb5kdc: starting...
W20260808 23:40:04.111052 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.411s user 0.002s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:40:04 dist-test-slave-5k54 krb5kdc[23532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:40:04.327960 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:33935
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:33935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:04.658087 23546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:04.658437 23546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:04.658617 23546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:04.665799 23546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:04.665995 23546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:04.666109 23546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:04.666213 23546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:04.666291 23546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:04.675915 23546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:33935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:33935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:04.677166 23546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:04.678599 23546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:04.690717 23555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:04.690783 23553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:04.690806 23552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:04.691761 23546 server_base.cc:1034] running on GCE node
I20260808 23:40:04.692615 23546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:04.694022 23546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:04.695274 23546 hybrid_clock.cc:648] HybridClock initialized: now 1786232404695233 us; error 101 us; skew 500 ppm
I20260808 23:40:04.695668 23546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:04 dist-test-slave-5k54 krb5kdc[23532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232404, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:04.699018 23546 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:40:04.700230 23546 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:04.700361 23546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:04.700711 23546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:04.703706 23546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "51e8628ed0c0479fab503e34d0ac74eb"
format_stamp: "Formatted at 2026-08-08 23:40:04 on dist-test-slave-5k54"
server_key: "0be13a5b9b60ba6682e2a79a8753f5e0"
server_key_iv: "07317a93997f5a75fa33393c07a997f1"
server_key_version: "encryptionkey@0"
I20260808 23:40:04.704517 23546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "51e8628ed0c0479fab503e34d0ac74eb"
format_stamp: "Formatted at 2026-08-08 23:40:04 on dist-test-slave-5k54"
server_key: "0be13a5b9b60ba6682e2a79a8753f5e0"
server_key_iv: "07317a93997f5a75fa33393c07a997f1"
server_key_version: "encryptionkey@0"
I20260808 23:40:04.709440 23546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:40:04.712821 23561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:04.714094 23546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:40:04.714287 23546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "51e8628ed0c0479fab503e34d0ac74eb"
format_stamp: "Formatted at 2026-08-08 23:40:04 on dist-test-slave-5k54"
server_key: "0be13a5b9b60ba6682e2a79a8753f5e0"
server_key_iv: "07317a93997f5a75fa33393c07a997f1"
server_key_version: "encryptionkey@0"
I20260808 23:40:04.714511 23546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:04.737598 23546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:04.741739 23546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:04.742029 23546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:04.760661 23546 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:33935
I20260808 23:40:04.760669 23623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:33935 every 8 connection(s)
I20260808 23:40:04.761559 23546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:04.765251 23624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:04.768409 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 23546
I20260808 23:40:04.768656 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:04.769503 19178 external_mini_cluster.cc:1428] Setting key 21cb1071b14a904ca8c88db0ad79dfca
I20260808 23:40:04.778628 23624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb: Bootstrap starting.
I20260808 23:40:04.782570 23624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:04.784011 23624 log.cc:824] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:40:04 dist-test-slave-5k54 krb5kdc[23532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:04.787225 23624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb: No bootstrap required, opened a new log
I20260808 23:40:04.793426 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:04.771753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.18.186.190:33935)
0808 23:40:04.772582 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:04.772594 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:04.772634 (+ 40us) server_negotiation.cc:407] Connection header received
0808 23:40:04.774350 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:04.774372 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:04.774769 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:04.775113 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:04.777086 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:04.778351 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:04.780184 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:04.780884 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:04.781014 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:04.786229 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:04.786237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:04.786251 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:04.786284 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:04.789429 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:04.790239 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:04.790244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:04.790249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:04.790433 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:04.790814 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:04.790817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:04.790819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:04.791317 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:04.791601 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:04.792011 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 23:40:04.792383 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":373,"threads_started":1}
I20260808 23:40:04.796546 23624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e8628ed0c0479fab503e34d0ac74eb" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33935 } }
I20260808 23:40:04.796904 23624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:04.797013 23624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51e8628ed0c0479fab503e34d0ac74eb, State: Initialized, Role: FOLLOWER
I20260808 23:40:04.797564 23624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e8628ed0c0479fab503e34d0ac74eb" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33935 } }
I20260808 23:40:04.797847 23624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:04.797992 23624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:04.798174 23624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:04.800369 23624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e8628ed0c0479fab503e34d0ac74eb" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33935 } }
I20260808 23:40:04.800853 23624 leader_election.cc:302] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51e8628ed0c0479fab503e34d0ac74eb; no voters:
I20260808 23:40:04.801395 23624 leader_election.cc:288] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:04.801519 23629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:04.802793 23629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [term 1 LEADER]: Becoming Leader. State: Replica: 51e8628ed0c0479fab503e34d0ac74eb, State: Running, Role: LEADER
I20260808 23:40:04.803337 23629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e8628ed0c0479fab503e34d0ac74eb" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33935 } }
I20260808 23:40:04.803695 23624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:04.806210 23632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51e8628ed0c0479fab503e34d0ac74eb. Latest consensus state: current_term: 1 leader_uuid: "51e8628ed0c0479fab503e34d0ac74eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e8628ed0c0479fab503e34d0ac74eb" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33935 } } }
I20260808 23:40:04.806644 23632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:04.806730 23631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51e8628ed0c0479fab503e34d0ac74eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e8628ed0c0479fab503e34d0ac74eb" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 33935 } } }
I20260808 23:40:04.807168 23631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:04.810369 23640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:04.815790 23640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:04.824985 23640 catalog_manager.cc:1347] Generated new cluster ID: da0e1c4657c94d30aff3ed7703524e62
I20260808 23:40:04.825126 23640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:04.841579 23640 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:04.842880 23640 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:04.853744 23640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb: Generated new TSK 0
I20260808 23:40:04.854664 23640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:40:04.926518 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:33935
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:05.270517 23658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:05.270848 23658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:05.271039 23658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:05.278178 23658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:05.278332 23658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:05.278527 23658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:05.288477 23658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:33935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:05.289767 23658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:05.291105 23658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:05.304205 23663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:05.304262 23666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:05.304308 23664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:05.305483 23658 server_base.cc:1034] running on GCE node
I20260808 23:40:05.306056 23658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:05.306989 23658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:05.308207 23658 hybrid_clock.cc:648] HybridClock initialized: now 1786232405308161 us; error 68 us; skew 500 ppm
I20260808 23:40:05.308545 23658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:05 dist-test-slave-5k54 krb5kdc[23532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232405, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:05.315277 23658 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:40:05.316514 23658 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:05.316620 23658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:05.316963 23658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:05.320057 23658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "9b43b947d0c74ae3b42219049a2f15d7"
format_stamp: "Formatted at 2026-08-08 23:40:05 on dist-test-slave-5k54"
server_key: "ad6035d0391f08f15d4aaeebf1875950"
server_key_iv: "bcdb176e75333f3af2f1eec53338d72f"
server_key_version: "encryptionkey@0"
I20260808 23:40:05.320860 23658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "9b43b947d0c74ae3b42219049a2f15d7"
format_stamp: "Formatted at 2026-08-08 23:40:05 on dist-test-slave-5k54"
server_key: "ad6035d0391f08f15d4aaeebf1875950"
server_key_iv: "bcdb176e75333f3af2f1eec53338d72f"
server_key_version: "encryptionkey@0"
I20260808 23:40:05.325783 23658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260808 23:40:05.329175 23672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:05.330451 23658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 23:40:05.330613 23658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "9b43b947d0c74ae3b42219049a2f15d7"
format_stamp: "Formatted at 2026-08-08 23:40:05 on dist-test-slave-5k54"
server_key: "ad6035d0391f08f15d4aaeebf1875950"
server_key_iv: "bcdb176e75333f3af2f1eec53338d72f"
server_key_version: "encryptionkey@0"
I20260808 23:40:05.330811 23658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:05.354455 23658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:05.358740 23658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:05.358973 23658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:05.359910 23658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:05.361169 23658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:05.361259 23658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:05.361369 23658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:05.361472 23658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:05.393523 23658 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:40319
I20260808 23:40:05.393613 23793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:40319 every 8 connection(s)
I20260808 23:40:05.394467 23658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:05.402580 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 23658
I20260808 23:40:05.402830 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:40:05.403704 19178 external_mini_cluster.cc:1428] Setting key 874a1ffa133522db776084c1dbad737a
Aug 08 23:40:05 dist-test-slave-5k54 krb5kdc[23532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232405, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:05.414867 23797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:05.396915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:47581 (local address 127.18.186.190:33935)
0808 23:40:05.397402 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:05.397408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:05.398195 (+ 787us) server_negotiation.cc:407] Connection header received
0808 23:40:05.400128 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:05.400133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:05.400228 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:05.400345 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:05.401793 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:05.402911 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:05.404920 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:05.405401 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:05.405540 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:05.409127 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:05.409132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:05.409136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:05.409198 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:05.411875 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:05.412596 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:05.412602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:05.412607 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:05.412729 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:05.413160 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:05.413172 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:05.413175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:05.413429 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:05.413583 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:05.414271 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 23:40:05.414510 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":264,"threads_started":1}
I20260808 23:40:05.415921 23798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:05.397355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:33935 (local address 127.18.186.129:47581)
0808 23:40:05.398045 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:05.398081 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:05.399858 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:05.400575 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:05.400583 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:05.401092 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:05.401630 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:05.401646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:05.403123 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:05.403129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:05.404756 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:05.404780 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:05.405649 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:05.405658 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:05.405937 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:05.406639 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:05.406662 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:05.408908 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:05.412026 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:05.412035 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:05.412051 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:05.412469 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:05.412844 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:05.412849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:05.412853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:05.413039 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:05.413556 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:05.413564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:05.413894 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 23:40:05.414161 (+ 267us) client_negotiation.cc:240] Negotiation successful
0808 23:40:05.414690 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":283,"threads_started":1}
I20260808 23:40:05.417637 23795 heartbeater.cc:347] Connected to a master server at 127.18.186.190:33935
I20260808 23:40:05.417946 23795 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:05.418692 23795 heartbeater.cc:511] Master 127.18.186.190:33935 requested a full tablet report, sending...
I20260808 23:40:05.421114 23582 ts_manager.cc:194] Registered new tserver with Master: 9b43b947d0c74ae3b42219049a2f15d7 (127.18.186.129:40319)
I20260808 23:40:05.423383 23582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:47581
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:40:05.455688 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:33935
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:05.785102 23801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:05.785377 23801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:05.785480 23801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:05.792722 23801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:05.792822 23801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:05.792940 23801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:05.802636 23801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:33935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:05.803892 23801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:05.805400 23801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:05.818657 23807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:05.818686 23808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:05.818730 23810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:05.819953 23801 server_base.cc:1034] running on GCE node
I20260808 23:40:05.820530 23801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:05.821426 23801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:05.822676 23801 hybrid_clock.cc:648] HybridClock initialized: now 1786232405822622 us; error 48 us; skew 500 ppm
I20260808 23:40:05.822999 23801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:05 dist-test-slave-5k54 krb5kdc[23532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232405, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:05.826162 23801 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:40:05.827217 23801 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:05.827304 23801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:05.827636 23801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:05.830380 23801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "2e4365e6a5534b92aaf4f43f5328f2ed"
format_stamp: "Formatted at 2026-08-08 23:40:05 on dist-test-slave-5k54"
server_key: "76db127c9a8b9c8deff9f6d58851e233"
server_key_iv: "90db75d57a0046fc8460eea999f5d11f"
server_key_version: "encryptionkey@0"
I20260808 23:40:05.831133 23801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "2e4365e6a5534b92aaf4f43f5328f2ed"
format_stamp: "Formatted at 2026-08-08 23:40:05 on dist-test-slave-5k54"
server_key: "76db127c9a8b9c8deff9f6d58851e233"
server_key_iv: "90db75d57a0046fc8460eea999f5d11f"
server_key_version: "encryptionkey@0"
I20260808 23:40:05.836287 23801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 23:40:05.839581 23817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:05.840819 23801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:40:05.840970 23801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "2e4365e6a5534b92aaf4f43f5328f2ed"
format_stamp: "Formatted at 2026-08-08 23:40:05 on dist-test-slave-5k54"
server_key: "76db127c9a8b9c8deff9f6d58851e233"
server_key_iv: "90db75d57a0046fc8460eea999f5d11f"
server_key_version: "encryptionkey@0"
I20260808 23:40:05.841140 23801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:05.860634 23801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:05.864459 23801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:05.864682 23801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:05.865595 23801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:05.866847 23801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:05.866941 23801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:05.867022 23801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:05.867077 23801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:05.900311 23801 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:35117
I20260808 23:40:05.900398 23945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:35117 every 8 connection(s)
I20260808 23:40:05.901176 23801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:05.907160 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 23801
I20260808 23:40:05.907375 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:40:05.907841 19178 external_mini_cluster.cc:1428] Setting key 5cf13856b0a1b6a7c5d3dcffa27bc819
Aug 08 23:40:05 dist-test-slave-5k54 krb5kdc[23532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232405, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:05.923166 23797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:05.903712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:60771 (local address 127.18.186.190:33935)
0808 23:40:05.903896 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:05.903901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:05.905227 (+ 1326us) server_negotiation.cc:407] Connection header received
0808 23:40:05.907061 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:05.907067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:05.907164 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:05.907277 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:05.909178 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:05.910132 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:05.911681 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:05.912429 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:05.912568 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:05.917021 (+ 4453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:05.917029 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:05.917037 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:05.917079 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:05.920003 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:05.920707 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:05.920715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:05.920721 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:05.920875 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:05.921263 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:05.921271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:05.921277 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:05.921668 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:05.921815 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:05.922473 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 23:40:05.922743 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:40:05.923753 23949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:05.904198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:33935 (local address 127.18.186.130:60771)
0808 23:40:05.905061 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:05.905105 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:05.906780 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:05.907553 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:05.907562 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:05.908158 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:05.908982 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:05.909006 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:05.910281 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:05.910285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:05.911543 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:05.911555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:05.912642 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:05.912650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:05.912926 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:05.913692 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:05.913720 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:05.916758 (+ 3038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:05.920133 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:05.920140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:05.920152 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:05.920569 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:05.920981 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:05.920984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:05.920986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:05.921151 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:05.921788 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:05.921793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:05.922083 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 23:40:05.922298 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 23:40:05.922660 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":442,"threads_started":1}
I20260808 23:40:05.925365 23946 heartbeater.cc:347] Connected to a master server at 127.18.186.190:33935
I20260808 23:40:05.925695 23946 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:05.926352 23946 heartbeater.cc:511] Master 127.18.186.190:33935 requested a full tablet report, sending...
I20260808 23:40:05.927914 23582 ts_manager.cc:194] Registered new tserver with Master: 2e4365e6a5534b92aaf4f43f5328f2ed (127.18.186.130:35117)
I20260808 23:40:05.929199 23582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:60771
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:40:05.961025 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:33935
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:06.285835 23953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:06.286124 23953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:06.286262 23953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:06.293390 23953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:06.293488 23953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:06.293642 23953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:06.303319 23953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:33935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:06.304670 23953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:06.306013 23953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:06.319154 23964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:06.319128 23960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:06.319458 23962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:06.319969 23953 server_base.cc:1034] running on GCE node
I20260808 23:40:06.320523 23953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:06.321513 23953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:06.322732 23953 hybrid_clock.cc:648] HybridClock initialized: now 1786232406322675 us; error 54 us; skew 500 ppm
I20260808 23:40:06.323047 23953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:06 dist-test-slave-5k54 krb5kdc[23532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232406, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:06.326213 23953 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:40:06.327378 23953 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:06.327535 23953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:06.327860 23953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:06.330768 23953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "29a3a65102324aa796967df1a014d062"
format_stamp: "Formatted at 2026-08-08 23:40:06 on dist-test-slave-5k54"
server_key: "1a1a363d4ec3b2e58564f6325b1e052a"
server_key_iv: "138db5c4986c7e1ed80ec1ae9ad7054f"
server_key_version: "encryptionkey@0"
I20260808 23:40:06.331709 23953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "29a3a65102324aa796967df1a014d062"
format_stamp: "Formatted at 2026-08-08 23:40:06 on dist-test-slave-5k54"
server_key: "1a1a363d4ec3b2e58564f6325b1e052a"
server_key_iv: "138db5c4986c7e1ed80ec1ae9ad7054f"
server_key_version: "encryptionkey@0"
I20260808 23:40:06.336776 23953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 23:40:06.340197 23970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:06.341553 23953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 23:40:06.341727 23953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "29a3a65102324aa796967df1a014d062"
format_stamp: "Formatted at 2026-08-08 23:40:06 on dist-test-slave-5k54"
server_key: "1a1a363d4ec3b2e58564f6325b1e052a"
server_key_iv: "138db5c4986c7e1ed80ec1ae9ad7054f"
server_key_version: "encryptionkey@0"
I20260808 23:40:06.341967 23953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:06.380928 23953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:06.387041 23953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:06.387336 23953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:06.388448 23953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:06.390254 23953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:06.390386 23953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:06.390502 23953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:06.390583 23953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:06.426545 23795 heartbeater.cc:503] Master 127.18.186.190:33935 was elected leader, sending a full tablet report...
I20260808 23:40:06.428725 23953 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:37657
I20260808 23:40:06.428797 24089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:37657 every 8 connection(s)
I20260808 23:40:06.429819 23953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:06.432794 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 23953
I20260808 23:40:06.433027 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:40:06.433447 19178 external_mini_cluster.cc:1428] Setting key 30301c1764e998cfaf4edc1871342f00
Aug 08 23:40:06 dist-test-slave-5k54 krb5kdc[23532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232406, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:06.451805 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.432722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:33977 (local address 127.18.186.190:33935)
0808 23:40:06.433213 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.433219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.434588 (+ 1369us) server_negotiation.cc:407] Connection header received
0808 23:40:06.437274 (+ 2686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.437280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.437380 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.437496 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:06.439031 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.439895 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.441462 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.441892 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.442025 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.445826 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:06.445831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:06.445834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:06.445866 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:06.448870 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:06.449603 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:06.449609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:06.449612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:06.449746 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:06.450152 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:06.450160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:06.450163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:06.450426 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:06.450586 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.451269 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.451535 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":256,"threads_started":1}
I20260808 23:40:06.452716 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.433457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:33935 (local address 127.18.186.131:33977)
0808 23:40:06.434415 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:06.434469 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:06.437003 (+ 2534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:06.437752 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:06.437760 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:06.438281 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:06.438852 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.438870 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.440055 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.440061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.441316 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.441332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.442128 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.442135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.442364 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.443090 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:06.443112 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:06.445583 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:06.449016 (+ 3433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:06.449025 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:06.449047 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:06.449474 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:06.449857 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:06.449862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:06.449867 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:06.450037 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:06.450559 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:06.450566 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:06.450889 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 23:40:06.451179 (+ 290us) client_negotiation.cc:240] Negotiation successful
0808 23:40:06.451625 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":404,"threads_started":1}
I20260808 23:40:06.454111 24091 heartbeater.cc:347] Connected to a master server at 127.18.186.190:33935
I20260808 23:40:06.454463 24091 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:06.455155 24091 heartbeater.cc:511] Master 127.18.186.190:33935 requested a full tablet report, sending...
I20260808 23:40:06.456576 23582 ts_manager.cc:194] Registered new tserver with Master: 29a3a65102324aa796967df1a014d062 (127.18.186.131:37657)
I20260808 23:40:06.457397 23582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:33977
I20260808 23:40:06.461192 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:40:06.477998 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.465699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.18.186.190:33935)
0808 23:40:06.465898 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.465908 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.466331 (+ 423us) server_negotiation.cc:407] Connection header received
0808 23:40:06.466538 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.466544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.466618 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.466717 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:06.468212 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.468966 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.470201 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.470625 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.470756 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.473022 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:06.473028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:06.473040 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:06.473070 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:06.475347 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:06.476022 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:06.476029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:06.476033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:06.476162 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:06.476528 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:06.476534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:06.476538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:06.476863 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:06.477143 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.477458 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.477680 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:40:06.481778 23582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:40:06.488669 23582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:06.522661 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.501260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.18.186.131:37657)
0808 23:40:06.513464 (+ 12204us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.513475 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.513520 (+ 45us) server_negotiation.cc:407] Connection header received
0808 23:40:06.513660 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.513671 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.514040 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.514325 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:06.515284 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.516869 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.519840 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.521358 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.521568 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.521804 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.522112 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.522285 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11954,"thread_start_us":12648,"threads_started":1}
I20260808 23:40:06.523054 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.500882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37657 (local address 127.0.0.1:49376)
0808 23:40:06.501697 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:06.501750 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:06.501937 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:06.514406 (+ 12469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:06.514419 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:06.514471 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:06.515063 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.515076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.517025 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.517031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.519649 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.519666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.521596 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.521605 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.521841 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.521911 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 23:40:06.522046 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 23:40:06.522253 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":370,"threads_started":1}
I20260808 23:40:06.524477 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.502113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:40319 (local address 127.0.0.1:54782)
0808 23:40:06.516320 (+ 14207us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:06.516342 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:06.516467 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:06.517105 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:06.517112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:06.517144 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:06.517583 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.517592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.520031 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.520042 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.521533 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.521549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.523591 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.523600 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.523777 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.523820 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 23:40:06.523898 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 23:40:06.524021 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14102,"mutex_wait_us":20,"thread_start_us":13132,"threads_started":1}
I20260808 23:40:06.524923 24108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.501826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:35117 (local address 127.0.0.1:35742)
0808 23:40:06.515570 (+ 13744us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:06.515590 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:06.515740 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:06.516580 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:06.516585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:06.516610 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:06.517031 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.517045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.518971 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.518977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.521749 (+ 2772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.521764 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.524110 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.524114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.524241 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.524262 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 23:40:06.524380 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 23:40:06.524522 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13568,"mutex_wait_us":31,"thread_start_us":13209,"threads_started":1}
I20260808 23:40:06.525473 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.502626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.18.186.129:40319)
0808 23:40:06.512900 (+ 10274us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.512910 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.516463 (+ 3553us) server_negotiation.cc:407] Connection header received
0808 23:40:06.516603 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.516610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.516828 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.517726 (+ 898us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:06.517936 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.519843 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.521752 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.523256 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.524681 (+ 1425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.524885 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.524965 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.525143 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10099,"thread_start_us":9944,"threads_started":1}
I20260808 23:40:06.525810 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.502551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.18.186.130:35117)
0808 23:40:06.509978 (+ 7427us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.509992 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.515703 (+ 5711us) server_negotiation.cc:407] Connection header received
0808 23:40:06.515908 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.515913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.516117 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.516320 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:06.517220 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.518800 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.521902 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.523673 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.523826 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.523989 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.524425 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.525465 (+ 1040us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7195,"thread_start_us":7061,"threads_started":1}
I20260808 23:40:06.527668 24022 tablet_service.cc:1467] Processing CreateTablet for tablet 6564644efd8d4e6eb6964cfcb97f22a2 (DEFAULT_TABLE table=test-table [id=eeeebc0f8f6c42ee958cd9e34bc80d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:06.529466 24022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6564644efd8d4e6eb6964cfcb97f22a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:06.529143 23723 tablet_service.cc:1467] Processing CreateTablet for tablet 6564644efd8d4e6eb6964cfcb97f22a2 (DEFAULT_TABLE table=test-table [id=eeeebc0f8f6c42ee958cd9e34bc80d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:06.529143 23874 tablet_service.cc:1467] Processing CreateTablet for tablet 6564644efd8d4e6eb6964cfcb97f22a2 (DEFAULT_TABLE table=test-table [id=eeeebc0f8f6c42ee958cd9e34bc80d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:06.530596 23723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6564644efd8d4e6eb6964cfcb97f22a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:06.530597 23874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6564644efd8d4e6eb6964cfcb97f22a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:06.541718 24111 tablet_bootstrap.cc:492] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Bootstrap starting.
I20260808 23:40:06.545437 24111 tablet_bootstrap.cc:654] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:06.545446 24112 tablet_bootstrap.cc:492] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Bootstrap starting.
I20260808 23:40:06.546969 24111 log.cc:824] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:06.549548 24112 tablet_bootstrap.cc:654] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:06.551112 24112 log.cc:824] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:06.551723 24113 tablet_bootstrap.cc:492] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Bootstrap starting.
I20260808 23:40:06.554433 24111 tablet_bootstrap.cc:492] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: No bootstrap required, opened a new log
I20260808 23:40:06.554751 24111 ts_tablet_manager.cc:1397] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260808 23:40:06.555565 24113 tablet_bootstrap.cc:654] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:06.555604 24112 tablet_bootstrap.cc:492] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: No bootstrap required, opened a new log
I20260808 23:40:06.555861 24112 ts_tablet_manager.cc:1397] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260808 23:40:06.556767 24113 log.cc:824] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:06.559782 24113 tablet_bootstrap.cc:492] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: No bootstrap required, opened a new log
I20260808 23:40:06.560041 24113 ts_tablet_manager.cc:1397] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 23:40:06.563642 24112 raft_consensus.cc:348] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.564131 24112 raft_consensus.cc:374] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:06.564226 24112 raft_consensus.cc:729] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b43b947d0c74ae3b42219049a2f15d7, State: Initialized, Role: FOLLOWER
I20260808 23:40:06.564841 24112 consensus_queue.cc:260] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.565371 24111 raft_consensus.cc:348] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.565783 24111 raft_consensus.cc:374] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:06.565898 24111 raft_consensus.cc:729] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a3a65102324aa796967df1a014d062, State: Initialized, Role: FOLLOWER
I20260808 23:40:06.566476 24111 consensus_queue.cc:260] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.566680 24112 ts_tablet_manager.cc:1428] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260808 23:40:06.574905 24113 raft_consensus.cc:348] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.575505 24113 raft_consensus.cc:374] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:06.575639 24113 raft_consensus.cc:729] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4365e6a5534b92aaf4f43f5328f2ed, State: Initialized, Role: FOLLOWER
I20260808 23:40:06.576192 24091 heartbeater.cc:503] Master 127.18.186.190:33935 was elected leader, sending a full tablet report...
I20260808 23:40:06.576231 24113 consensus_queue.cc:260] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.577011 24111 ts_tablet_manager.cc:1428] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Time spent starting tablet: real 0.022s user 0.011s sys 0.011s
I20260808 23:40:06.577445 23946 heartbeater.cc:503] Master 127.18.186.190:33935 was elected leader, sending a full tablet report...
I20260808 23:40:06.578055 24113 ts_tablet_manager.cc:1428] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
W20260808 23:40:06.646850 23796 tablet.cc:2369] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:40:06.653019 23947 tablet.cc:2369] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:40:06.681553 24092 tablet.cc:2369] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:40:06.778097 24119 raft_consensus.cc:482] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:06.778475 24119 raft_consensus.cc:504] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.780282 24119 leader_election.cc:288] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29a3a65102324aa796967df1a014d062 (127.18.186.131:37657), 9b43b947d0c74ae3b42219049a2f15d7 (127.18.186.129:40319)
I20260808 23:40:06.787528 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.781658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:38807 (local address 127.18.186.129:40319)
0808 23:40:06.781851 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.781858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.782119 (+ 261us) server_negotiation.cc:407] Connection header received
0808 23:40:06.782249 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.782256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.782342 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.782450 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:06.783103 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.783860 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.785744 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.786505 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.786651 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.786736 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.787104 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.787236 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:40:06.787564 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.780755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:56579 (local address 127.18.186.131:37657)
0808 23:40:06.781031 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.781041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.781545 (+ 504us) server_negotiation.cc:407] Connection header received
0808 23:40:06.781786 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.781791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.781874 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.781953 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:06.782809 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.783721 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.785784 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.786544 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.786740 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.786809 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.787185 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.787319 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:40:06.787660 24122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.781427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:40319 (local address 127.18.186.130:38807)
0808 23:40:06.782007 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:06.782021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:06.782121 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:06.782471 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:06.782474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:06.782494 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:06.782944 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.782956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.783999 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.784003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.785584 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.785596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.786742 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.786748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.786937 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.786974 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 23:40:06.787045 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 23:40:06.787176 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":57,"thread_start_us":322,"threads_started":1}
I20260808 23:40:06.787712 24121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.780690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37657 (local address 127.18.186.130:56579)
0808 23:40:06.781396 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:06.781419 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:06.781587 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:06.782183 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:06.782187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:06.782232 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:06.782633 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.782644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.783871 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.783875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.785594 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:06.785606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.786796 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.786802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:06.786995 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.787018 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:40:06.787132 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 23:40:06.787249 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":136,"thread_start_us":402,"threads_started":1}
I20260808 23:40:06.788614 23745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6564644efd8d4e6eb6964cfcb97f22a2" candidate_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b43b947d0c74ae3b42219049a2f15d7" is_pre_election: true
I20260808 23:40:06.788666 24043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6564644efd8d4e6eb6964cfcb97f22a2" candidate_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a3a65102324aa796967df1a014d062" is_pre_election: true
I20260808 23:40:06.789106 23745 raft_consensus.cc:2454] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e4365e6a5534b92aaf4f43f5328f2ed in term 0.
I20260808 23:40:06.789135 24043 raft_consensus.cc:2454] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e4365e6a5534b92aaf4f43f5328f2ed in term 0.
I20260808 23:40:06.789752 23822 leader_election.cc:302] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29a3a65102324aa796967df1a014d062, 2e4365e6a5534b92aaf4f43f5328f2ed; no voters:
I20260808 23:40:06.790123 24119 raft_consensus.cc:2781] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:06.790264 24119 raft_consensus.cc:482] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:06.790369 24119 raft_consensus.cc:3037] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:06.792644 24119 raft_consensus.cc:504] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.793210 24119 leader_election.cc:288] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [CANDIDATE]: Term 1 election: Requested vote from peers 29a3a65102324aa796967df1a014d062 (127.18.186.131:37657), 9b43b947d0c74ae3b42219049a2f15d7 (127.18.186.129:40319)
I20260808 23:40:06.793677 24043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6564644efd8d4e6eb6964cfcb97f22a2" candidate_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29a3a65102324aa796967df1a014d062"
I20260808 23:40:06.793751 23745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6564644efd8d4e6eb6964cfcb97f22a2" candidate_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b43b947d0c74ae3b42219049a2f15d7"
I20260808 23:40:06.793913 24043 raft_consensus.cc:3037] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:06.793946 23745 raft_consensus.cc:3037] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:06.797201 23745 raft_consensus.cc:2454] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4365e6a5534b92aaf4f43f5328f2ed in term 1.
I20260808 23:40:06.797201 24043 raft_consensus.cc:2454] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4365e6a5534b92aaf4f43f5328f2ed in term 1.
I20260808 23:40:06.797636 23822 leader_election.cc:302] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4365e6a5534b92aaf4f43f5328f2ed, 9b43b947d0c74ae3b42219049a2f15d7; no voters:
I20260808 23:40:06.797906 24119 raft_consensus.cc:2781] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:06.798348 24119 raft_consensus.cc:686] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 1 LEADER]: Becoming Leader. State: Replica: 2e4365e6a5534b92aaf4f43f5328f2ed, State: Running, Role: LEADER
I20260808 23:40:06.798851 24119 consensus_queue.cc:237] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } }
I20260808 23:40:06.803777 23582 catalog_manager.cc:5495] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e4365e6a5534b92aaf4f43f5328f2ed (127.18.186.130). New cstate: current_term: 1 leader_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e4365e6a5534b92aaf4f43f5328f2ed" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35117 } health_report { overall_health: HEALTHY } } }
I20260808 23:40:06.822618 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.817497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.18.186.130:35117)
0808 23:40:06.817689 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.817694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.817977 (+ 283us) server_negotiation.cc:407] Connection header received
0808 23:40:06.818142 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.818146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.818222 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.818344 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:06.818697 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.819510 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.820787 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.821179 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.821267 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.821669 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:06.822143 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:06.822263 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.822338 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.822409 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 23:40:06.830821 24043 raft_consensus.cc:1261] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 1 FOLLOWER]: Refusing update from remote peer 2e4365e6a5534b92aaf4f43f5328f2ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:06.830821 23745 raft_consensus.cc:1261] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 1 FOLLOWER]: Refusing update from remote peer 2e4365e6a5534b92aaf4f43f5328f2ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:06.831804 24119 consensus_queue.cc:1035] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b43b947d0c74ae3b42219049a2f15d7" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 40319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:06.832144 24123 consensus_queue.cc:1035] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a3a65102324aa796967df1a014d062" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:06.849323 24130 mvcc.cc:205] Tried to move back new op lower bound from 7316407938368344064 to 7316407938252189696. Current Snapshot: MvccSnapshot[applied={T|T < 7316407938368344064}]
I20260808 23:40:06.858458 23581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:06.858714 23581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:06.862404 23581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51e8628ed0c0479fab503e34d0ac74eb: Sending DeleteTablet for 3 replicas of tablet 6564644efd8d4e6eb6964cfcb97f22a2
I20260808 23:40:06.863605 23874 tablet_service.cc:1514] Processing DeleteTablet for tablet 6564644efd8d4e6eb6964cfcb97f22a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:06 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:35742
I20260808 23:40:06.863626 23723 tablet_service.cc:1514] Processing DeleteTablet for tablet 6564644efd8d4e6eb6964cfcb97f22a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:06 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:54782
I20260808 23:40:06.863726 24022 tablet_service.cc:1514] Processing DeleteTablet for tablet 6564644efd8d4e6eb6964cfcb97f22a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:06 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:49376
I20260808 23:40:06.864645 24141 tablet_replica.cc:330] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: stopping tablet replica
I20260808 23:40:06.865116 24141 raft_consensus.cc:2229] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:06.865653 24141 raft_consensus.cc:2258] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:06.865747 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 23658
I20260808 23:40:06.867594 24141 ts_tablet_manager.cc:1905] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:06.874806 24141 ts_tablet_manager.cc:1918] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:06.875021 24141 log.cc:1196] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/6564644efd8d4e6eb6964cfcb97f22a2
I20260808 23:40:06.875623 24141 ts_tablet_manager.cc:1939] T 6564644efd8d4e6eb6964cfcb97f22a2 P 29a3a65102324aa796967df1a014d062: Deleting consensus metadata
I20260808 23:40:06.877655 23567 catalog_manager.cc:4841] TS 29a3a65102324aa796967df1a014d062 (127.18.186.131:37657): tablet 6564644efd8d4e6eb6964cfcb97f22a2 (table test-table [id=eeeebc0f8f6c42ee958cd9e34bc80d11]) successfully deleted
I20260808 23:40:06.884930 24140 tablet_replica.cc:330] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: stopping tablet replica
I20260808 23:40:06.885499 24140 raft_consensus.cc:2229] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:40:06.886284 24140 raft_consensus.cc:2258] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:06.887187 24138 tablet_replica.cc:330] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: stopping tablet replica
I20260808 23:40:06.887693 24138 raft_consensus.cc:2229] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:06.888216 24138 raft_consensus.cc:2258] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:06.889156 24140 ts_tablet_manager.cc:1905] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:06.890565 24138 ts_tablet_manager.cc:1905] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 23:40:06 dist-test-slave-5k54 krb5kdc[23532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:40:06.896826 24140 ts_tablet_manager.cc:1918] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:06.896996 24140 log.cc:1196] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/6564644efd8d4e6eb6964cfcb97f22a2
I20260808 23:40:06.897614 24140 ts_tablet_manager.cc:1939] T 6564644efd8d4e6eb6964cfcb97f22a2 P 2e4365e6a5534b92aaf4f43f5328f2ed: Deleting consensus metadata
I20260808 23:40:06.898190 24138 ts_tablet_manager.cc:1918] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:06.898411 24138 log.cc:1196] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/6564644efd8d4e6eb6964cfcb97f22a2
I20260808 23:40:06.899052 24138 ts_tablet_manager.cc:1939] T 6564644efd8d4e6eb6964cfcb97f22a2 P 9b43b947d0c74ae3b42219049a2f15d7: Deleting consensus metadata
I20260808 23:40:06.899459 23565 catalog_manager.cc:4841] TS 2e4365e6a5534b92aaf4f43f5328f2ed (127.18.186.130:35117): tablet 6564644efd8d4e6eb6964cfcb97f22a2 (table test-table [id=eeeebc0f8f6c42ee958cd9e34bc80d11]) successfully deleted
I20260808 23:40:06.900903 23567 catalog_manager.cc:4841] TS 9b43b947d0c74ae3b42219049a2f15d7 (127.18.186.129:40319): tablet 6564644efd8d4e6eb6964cfcb97f22a2 (table test-table [id=eeeebc0f8f6c42ee958cd9e34bc80d11]) successfully deleted
I20260808 23:40:06.904217 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:06.868082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.18.186.129:40319)
0808 23:40:06.868313 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:06.868322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:06.884057 (+ 15735us) server_negotiation.cc:407] Connection header received
0808 23:40:06.884252 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:06.884265 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:06.884425 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:06.884546 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:06.886972 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.888214 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.890158 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:06.890804 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:06.890904 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:06.896547 (+ 5643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:06.896562 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:06.896579 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:06.896621 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:06.900655 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:06.902050 (+ 1395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:06.902061 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:06.902065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:06.902204 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:06.902597 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:06.902602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:06.902605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:06.902937 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:06.903256 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:06.903637 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 23:40:06.903853 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:40:06.904911 23789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:06.994400 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 23658
I20260808 23:40:07.014613 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 23801
Aug 08 23:40:07 dist-test-slave-5k54 krb5kdc[23532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:40:07.031814 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:07.015470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.18.186.130:35117)
0808 23:40:07.015681 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:07.015687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:07.015708 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:40:07.015806 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:07.015812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:07.015899 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:07.016013 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:07.017135 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:07.018434 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:07.019852 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:07.020536 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:07.020659 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:07.024730 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:07.024748 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:07.024780 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:07.024831 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:07.028871 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:07.029530 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:07.029540 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:07.029543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:07.029714 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:07.030093 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:07.030098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:07.030101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:07.030612 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:07.030953 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:07.031118 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 23:40:07.031342 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:40:07.032663 23941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:07.146441 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 23801
I20260808 23:40:07.166200 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 23953
Aug 08 23:40:07 dist-test-slave-5k54 krb5kdc[23532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:40:07.180886 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:07.166973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49380 (local address 127.18.186.131:37657)
0808 23:40:07.167170 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:07.167176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:07.167197 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:40:07.167417 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:07.167452 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:07.167535 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:07.167653 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:07.168946 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:07.169677 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:07.170951 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:07.171361 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:07.171549 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:07.175768 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:07.175781 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:07.175797 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:07.175831 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:07.178445 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:07.179015 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:07.179022 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:07.179024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:07.179164 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:07.179595 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:07.179599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:07.179601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:07.179930 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:07.180143 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:07.180354 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 23:40:07.180521 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:40:07.181622 24085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:07.270716 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 23953
I20260808 23:40:07.289870 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 23546
I20260808 23:40:07.291025 23618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:07.399199 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 23546
2026-08-08T23:40:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4746 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T23:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:07Z Disabled control of system clock
I20260808 23:40:07.473096 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42135
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:40:07.795544 24150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:07.795846 24150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:07.795938 24150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:07.803045 24150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:07.803151 24150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:07.803189 24150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:07.803246 24150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:07.803290 24150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:07.812454 24150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42135
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:07.813864 24150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:07.815770 24150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:07.827462 24155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:07.827442 24156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:07.827732 24158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:07.828455 24150 server_base.cc:1034] running on GCE node
I20260808 23:40:07.829569 24150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:07.831032 24150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:07.832242 24150 hybrid_clock.cc:648] HybridClock initialized: now 1786232407832203 us; error 28 us; skew 500 ppm
I20260808 23:40:07.832603 24150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:07.834798 24150 webserver.cc:460] Webserver started at http://127.18.186.190:36817/ using document root <none> and password file <none>
I20260808 23:40:07.835583 24150 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:07.835702 24150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:07.836068 24150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:07.838939 24150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "575e36693d834b48a79dbcab9e8e1ed4"
format_stamp: "Formatted at 2026-08-08 23:40:07 on dist-test-slave-5k54"
server_key: "c3e7f0869c301fe03028ad5824e1aab1"
server_key_iv: "5442791088c1212f56c5cc0029379a2c"
server_key_version: "encryptionkey@0"
I20260808 23:40:07.839808 24150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "575e36693d834b48a79dbcab9e8e1ed4"
format_stamp: "Formatted at 2026-08-08 23:40:07 on dist-test-slave-5k54"
server_key: "c3e7f0869c301fe03028ad5824e1aab1"
server_key_iv: "5442791088c1212f56c5cc0029379a2c"
server_key_version: "encryptionkey@0"
I20260808 23:40:07.844937 24150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:40:07.848292 24164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:07.849597 24150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:40:07.849784 24150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "575e36693d834b48a79dbcab9e8e1ed4"
format_stamp: "Formatted at 2026-08-08 23:40:07 on dist-test-slave-5k54"
server_key: "c3e7f0869c301fe03028ad5824e1aab1"
server_key_iv: "5442791088c1212f56c5cc0029379a2c"
server_key_version: "encryptionkey@0"
I20260808 23:40:07.850014 24150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:07.855754 24150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:07.856547 24150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:07.856861 24150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:07.875257 24150 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42135
I20260808 23:40:07.875289 24219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42135 every 8 connection(s)
I20260808 23:40:07.876633 24150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:07.880414 24220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:07.882201 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 24150
I20260808 23:40:07.882407 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:07.883245 19178 external_mini_cluster.cc:1428] Setting key e9cddaacb61a35ca1a0287720ecb809b
I20260808 23:40:07.891692 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:07.885367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.18.186.190:42135)
0808 23:40:07.886422 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:07.886435 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:07.886474 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:40:07.888148 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:07.888169 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:07.888566 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:07.888891 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:07.890031 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:07.890037 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:07.890053 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:07.890067 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:07.890136 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:07.890151 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:07.890325 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:07.890441 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 23:40:07.890750 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":412,"threads_started":1}
I20260808 23:40:07.892830 24220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4: Bootstrap starting.
I20260808 23:40:07.896207 24220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:07.897128 24220 log.cc:824] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:07.899875 24220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4: No bootstrap required, opened a new log
I20260808 23:40:07.906806 24220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575e36693d834b48a79dbcab9e8e1ed4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42135 } }
I20260808 23:40:07.907146 24220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:07.907250 24220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575e36693d834b48a79dbcab9e8e1ed4, State: Initialized, Role: FOLLOWER
I20260808 23:40:07.907887 24220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575e36693d834b48a79dbcab9e8e1ed4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42135 } }
I20260808 23:40:07.908116 24220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:07.908257 24220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:07.908447 24220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:07.910631 24220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575e36693d834b48a79dbcab9e8e1ed4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42135 } }
I20260808 23:40:07.911072 24220 leader_election.cc:302] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 575e36693d834b48a79dbcab9e8e1ed4; no voters:
I20260808 23:40:07.911682 24220 leader_election.cc:288] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:07.911866 24225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:07.912598 24225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [term 1 LEADER]: Becoming Leader. State: Replica: 575e36693d834b48a79dbcab9e8e1ed4, State: Running, Role: LEADER
I20260808 23:40:07.913197 24225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575e36693d834b48a79dbcab9e8e1ed4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42135 } }
I20260808 23:40:07.913825 24220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:07.915877 24227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 575e36693d834b48a79dbcab9e8e1ed4. Latest consensus state: current_term: 1 leader_uuid: "575e36693d834b48a79dbcab9e8e1ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575e36693d834b48a79dbcab9e8e1ed4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42135 } } }
I20260808 23:40:07.916206 24227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:07.917204 24226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "575e36693d834b48a79dbcab9e8e1ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "575e36693d834b48a79dbcab9e8e1ed4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42135 } } }
I20260808 23:40:07.917526 24226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:07.920560 24234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:07.925818 24234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:07.934571 24234 catalog_manager.cc:1347] Generated new cluster ID: 27b2e23d73f746c792c5f7632c63ea8f
I20260808 23:40:07.934692 24234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:07.952848 24234 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:07.953316 24234 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:07.969374 24234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4: Generated new TSK 0
I20260808 23:40:07.970194 24234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:40:07.976919 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42135
--builtin_ntp_servers=127.18.186.148:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:40:08.329516 24247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:08.329789 24247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:08.329941 24247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:08.337177 24247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:08.337281 24247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:08.337488 24247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:08.346745 24247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:08.348124 24247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:08.349849 24247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:08.362500 24253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:08.362534 24254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:08.362571 24256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:08.363698 24247 server_base.cc:1034] running on GCE node
I20260808 23:40:08.364234 24247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:08.365154 24247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:08.366349 24247 hybrid_clock.cc:648] HybridClock initialized: now 1786232408366309 us; error 30 us; skew 500 ppm
I20260808 23:40:08.366668 24247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:08.368906 24247 webserver.cc:460] Webserver started at http://127.18.186.129:45847/ using document root <none> and password file <none>
I20260808 23:40:08.369601 24247 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:08.369721 24247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:08.370071 24247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:08.372938 24247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "7e20ad4509c44bbdb83bccb6e68a206a"
format_stamp: "Formatted at 2026-08-08 23:40:08 on dist-test-slave-5k54"
server_key: "70edc568d4bca68e2636a215742d8fa5"
server_key_iv: "e7d6fa53a0609a10bd8210989e921147"
server_key_version: "encryptionkey@0"
I20260808 23:40:08.373741 24247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "7e20ad4509c44bbdb83bccb6e68a206a"
format_stamp: "Formatted at 2026-08-08 23:40:08 on dist-test-slave-5k54"
server_key: "70edc568d4bca68e2636a215742d8fa5"
server_key_iv: "e7d6fa53a0609a10bd8210989e921147"
server_key_version: "encryptionkey@0"
I20260808 23:40:08.378716 24247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:40:08.382159 24262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:08.383492 24247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:40:08.383689 24247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "7e20ad4509c44bbdb83bccb6e68a206a"
format_stamp: "Formatted at 2026-08-08 23:40:08 on dist-test-slave-5k54"
server_key: "70edc568d4bca68e2636a215742d8fa5"
server_key_iv: "e7d6fa53a0609a10bd8210989e921147"
server_key_version: "encryptionkey@0"
I20260808 23:40:08.383970 24247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:08.389523 24247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:08.390316 24247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:08.390599 24247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:08.391567 24247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:08.392956 24247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:08.393079 24247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:08.393215 24247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:08.393316 24247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:08.426309 24247 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:39153
I20260808 23:40:08.426357 24386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:39153 every 8 connection(s)
I20260808 23:40:08.427825 24247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:08.431661 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 24247
I20260808 23:40:08.431852 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:40:08.432531 19178 external_mini_cluster.cc:1428] Setting key 5ac7ef42fe968ca40c1c883f5e07a58f
I20260808 23:40:08.435585 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42135
--builtin_ntp_servers=127.18.186.148:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 23:40:08.437176 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:08.430575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:47437 (local address 127.18.186.190:42135)
0808 23:40:08.431011 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:08.431015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:08.432542 (+ 1527us) server_negotiation.cc:407] Connection header received
0808 23:40:08.434726 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:08.434732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:08.434802 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:08.434926 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:08.436280 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:08.436290 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:08.436298 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:08.436322 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:08.436381 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:08.436399 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:08.436535 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:08.436701 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 23:40:08.436770 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":230,"threads_started":1}
I20260808 23:40:08.437901 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:08.431225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42135 (local address 127.18.186.129:47437)
0808 23:40:08.432355 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:08.432407 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:08.434428 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:08.435124 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:08.435132 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:08.435224 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:08.435983 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:08.435986 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:08.436024 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:08.436109 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:08.436504 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:08.436511 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:08.436517 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 23:40:08.436725 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 23:40:08.436931 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":388,"threads_started":1}
I20260808 23:40:08.439389 24387 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42135
I20260808 23:40:08.439700 24387 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:08.440263 24387 heartbeater.cc:511] Master 127.18.186.190:42135 requested a full tablet report, sending...
I20260808 23:40:08.442255 24182 ts_manager.cc:194] Registered new tserver with Master: 7e20ad4509c44bbdb83bccb6e68a206a (127.18.186.129:39153)
W20260808 23:40:08.762817 24392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:08.763090 24392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:08.763235 24392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:08.770486 24392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:08.770601 24392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:08.770780 24392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:08.779659 24392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:08.780910 24392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:08.782618 24392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:08.795979 24397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:08.796028 24398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:08.796053 24400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:08.796777 24392 server_base.cc:1034] running on GCE node
I20260808 23:40:08.797322 24392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:08.798204 24392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:08.799397 24392 hybrid_clock.cc:648] HybridClock initialized: now 1786232408799339 us; error 53 us; skew 500 ppm
I20260808 23:40:08.799752 24392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:08.802006 24392 webserver.cc:460] Webserver started at http://127.18.186.130:35849/ using document root <none> and password file <none>
I20260808 23:40:08.802665 24392 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:08.802775 24392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:08.803128 24392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:08.805974 24392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "5dd1773661e84de1a8227de95b60c14a"
format_stamp: "Formatted at 2026-08-08 23:40:08 on dist-test-slave-5k54"
server_key: "93bd80efd4ffdc5c456c3a5d30df908f"
server_key_iv: "92fb0f8056d390379b9d4b6692b50be8"
server_key_version: "encryptionkey@0"
I20260808 23:40:08.806742 24392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "5dd1773661e84de1a8227de95b60c14a"
format_stamp: "Formatted at 2026-08-08 23:40:08 on dist-test-slave-5k54"
server_key: "93bd80efd4ffdc5c456c3a5d30df908f"
server_key_iv: "92fb0f8056d390379b9d4b6692b50be8"
server_key_version: "encryptionkey@0"
I20260808 23:40:08.811703 24392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 23:40:08.815135 24406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:08.816603 24392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:08.816768 24392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "5dd1773661e84de1a8227de95b60c14a"
format_stamp: "Formatted at 2026-08-08 23:40:08 on dist-test-slave-5k54"
server_key: "93bd80efd4ffdc5c456c3a5d30df908f"
server_key_iv: "92fb0f8056d390379b9d4b6692b50be8"
server_key_version: "encryptionkey@0"
I20260808 23:40:08.816989 24392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:08.822850 24392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:08.823705 24392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:08.823982 24392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:08.824892 24392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:08.826349 24392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:08.826462 24392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:08.826596 24392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:08.826686 24392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:08.859964 24392 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:36129
I20260808 23:40:08.860087 24528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:36129 every 8 connection(s)
I20260808 23:40:08.861347 24392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:08.867121 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 24392
I20260808 23:40:08.867313 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:40:08.867921 19178 external_mini_cluster.cc:1428] Setting key b997aac5fed5f6766f4610771af5baa5
I20260808 23:40:08.869760 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:08.863895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:36643 (local address 127.18.186.190:42135)
0808 23:40:08.864071 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:08.864076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:08.865421 (+ 1345us) server_negotiation.cc:407] Connection header received
0808 23:40:08.867226 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:08.867233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:08.867303 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:08.867457 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:08.868955 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:08.868962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:08.868969 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:08.868992 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:08.869035 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:08.869054 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:08.869183 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:08.869424 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 23:40:08.869482 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:40:08.871031 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:08.864390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42135 (local address 127.18.186.130:36643)
0808 23:40:08.865293 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:08.865328 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:08.866912 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:08.867721 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:08.867733 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:08.867853 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:08.868579 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:08.868581 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:08.868619 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:08.868704 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:08.869178 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:08.869191 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:08.869198 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:40:08.869509 (+ 311us) client_negotiation.cc:240] Negotiation successful
0808 23:40:08.869778 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":284,"threads_started":1}
I20260808 23:40:08.871678 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42135
--builtin_ntp_servers=127.18.186.148:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 23:40:08.872959 24529 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42135
I20260808 23:40:08.873296 24529 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:08.873950 24529 heartbeater.cc:511] Master 127.18.186.190:42135 requested a full tablet report, sending...
I20260808 23:40:08.875522 24182 ts_manager.cc:194] Registered new tserver with Master: 5dd1773661e84de1a8227de95b60c14a (127.18.186.130:36129)
W20260808 23:40:09.203112 24532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:09.203394 24532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:09.203621 24532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:09.210801 24532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:09.210909 24532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:09.211108 24532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:09.220652 24532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:09.222259 24532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:09.224421 24532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:09.237332 24538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:09.237360 24539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:09.237290 24541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:09.238101 24532 server_base.cc:1034] running on GCE node
I20260808 23:40:09.238682 24532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:09.239604 24532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:09.240805 24532 hybrid_clock.cc:648] HybridClock initialized: now 1786232409240771 us; error 32 us; skew 500 ppm
I20260808 23:40:09.241144 24532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:09.244014 24532 webserver.cc:460] Webserver started at http://127.18.186.131:32829/ using document root <none> and password file <none>
I20260808 23:40:09.244719 24532 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:09.244848 24532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:09.245206 24532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:09.248080 24532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "1daf0c025833426f95adc8dadf9c4156"
format_stamp: "Formatted at 2026-08-08 23:40:09 on dist-test-slave-5k54"
server_key: "4527f5d1976d487d9a4f6a556f80f182"
server_key_iv: "6247cc6d660bb8f0281273f084d20b73"
server_key_version: "encryptionkey@0"
I20260808 23:40:09.248869 24532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "1daf0c025833426f95adc8dadf9c4156"
format_stamp: "Formatted at 2026-08-08 23:40:09 on dist-test-slave-5k54"
server_key: "4527f5d1976d487d9a4f6a556f80f182"
server_key_iv: "6247cc6d660bb8f0281273f084d20b73"
server_key_version: "encryptionkey@0"
I20260808 23:40:09.254047 24532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:40:09.257779 24547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:09.259027 24532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 23:40:09.259191 24532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "1daf0c025833426f95adc8dadf9c4156"
format_stamp: "Formatted at 2026-08-08 23:40:09 on dist-test-slave-5k54"
server_key: "4527f5d1976d487d9a4f6a556f80f182"
server_key_iv: "6247cc6d660bb8f0281273f084d20b73"
server_key_version: "encryptionkey@0"
I20260808 23:40:09.259418 24532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:09.264899 24532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:09.265681 24532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:09.265949 24532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:09.266860 24532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:09.268271 24532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:09.268383 24532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:09.268514 24532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:09.268610 24532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:09.301729 24532 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:41855
I20260808 23:40:09.301803 24669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:41855 every 8 connection(s)
I20260808 23:40:09.303083 24532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:09.307695 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 24532
I20260808 23:40:09.307862 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:40:09.308313 19178 external_mini_cluster.cc:1428] Setting key 6f0ddffbbd476257b065407f45aadba8
I20260808 23:40:09.311791 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.305858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:48001 (local address 127.18.186.190:42135)
0808 23:40:09.306047 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.306051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.307480 (+ 1429us) server_negotiation.cc:407] Connection header received
0808 23:40:09.309394 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.309406 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.309488 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.309608 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.310990 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.310996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.311000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.311020 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.311061 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.311079 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.311219 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.311451 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.311521 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:40:09.312666 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.306340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42135 (local address 127.18.186.131:48001)
0808 23:40:09.307297 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.307339 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:09.309153 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:09.309845 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:09.309853 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:09.309952 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:09.310649 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:09.310652 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:09.310692 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:09.310822 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:09.311200 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:09.311211 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:09.311219 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:40:09.311521 (+ 302us) client_negotiation.cc:240] Negotiation successful
0808 23:40:09.311763 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":269,"threads_started":1}
I20260808 23:40:09.313894 24671 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42135
I20260808 23:40:09.314179 24671 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:09.314814 24671 heartbeater.cc:511] Master 127.18.186.190:42135 requested a full tablet report, sending...
I20260808 23:40:09.316154 24182 ts_manager.cc:194] Registered new tserver with Master: 1daf0c025833426f95adc8dadf9c4156 (127.18.186.131:41855)
I20260808 23:40:09.323769 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:40:09.330698 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.328296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40138 (local address 127.18.186.190:42135)
0808 23:40:09.328502 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.328510 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.329029 (+ 519us) server_negotiation.cc:407] Connection header received
0808 23:40:09.329178 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.329182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.329232 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.329312 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.330151 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.330155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.330157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.330172 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.330202 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.330214 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.330305 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.330338 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.330382 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:40:09.334649 24182 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:40:09.341401 24182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:09.370352 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.354037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.18.186.129:39153)
0808 23:40:09.367106 (+ 13069us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.367115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.367194 (+ 79us) server_negotiation.cc:407] Connection header received
0808 23:40:09.367272 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.367282 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.367567 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.368321 (+ 754us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.369413 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.369435 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.369453 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.369472 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.369544 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.369570 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.369712 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.369908 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.370022 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12905,"thread_start_us":13054,"threads_started":1}
I20260808 23:40:09.370455 24684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.353491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39153 (local address 127.0.0.1:51572)
0808 23:40:09.366524 (+ 13033us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.366541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:09.366631 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:09.367762 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:09.367769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:09.367791 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:09.369177 (+ 1386us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:09.369180 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:09.369194 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:09.369237 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:09.369693 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:09.369707 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:09.369716 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 23:40:09.369926 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 23:40:09.370016 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12895,"thread_start_us":13928,"threads_started":1}
I20260808 23:40:09.372280 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.353248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:36129 (local address 127.0.0.1:42736)
0808 23:40:09.365057 (+ 11809us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.365122 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:09.365289 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:09.367571 (+ 2282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:09.367581 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:09.367715 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:09.368918 (+ 1203us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:09.368922 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:09.368951 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:09.369038 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:09.371835 (+ 2797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:09.371840 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:09.371845 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:40:09.371974 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 23:40:09.372014 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11461,"thread_start_us":12991,"threads_started":1}
I20260808 23:40:09.372974 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.353456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.18.186.130:36129)
0808 23:40:09.366333 (+ 12877us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.366346 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.366385 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:40:09.366481 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.366492 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.366777 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.368113 (+ 1336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.371266 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.371274 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.371286 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.371302 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.371352 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.371379 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.372551 (+ 1172us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.372618 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.372724 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12669,"thread_start_us":13453,"threads_started":1}
I20260808 23:40:09.373693 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.354028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:41855 (local address 127.0.0.1:33734)
0808 23:40:09.367686 (+ 13658us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.367705 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:09.367781 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:09.371217 (+ 3436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:09.371220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:09.371238 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:09.372601 (+ 1363us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:09.372604 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:09.372620 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:09.372662 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:09.373065 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:09.373072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:09.373077 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:40:09.373213 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 23:40:09.373261 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13522,"thread_start_us":11412,"threads_started":1}
I20260808 23:40:09.373968 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.354573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.18.186.131:41855)
0808 23:40:09.369727 (+ 15154us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.369737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.369767 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:40:09.369857 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.369864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.370131 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.370275 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.372823 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.372832 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.372846 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.372865 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.372925 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.372944 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.373279 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.373354 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.373453 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14970,"thread_start_us":14793,"threads_started":1}
I20260808 23:40:09.374648 24313 tablet_service.cc:1467] Processing CreateTablet for tablet a11df4e275f64325987e10de30423ffe (DEFAULT_TABLE table=test-table [id=717f8bac3640407bb2581c4cdc4ebf22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:09.376068 24313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11df4e275f64325987e10de30423ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:09.377002 24458 tablet_service.cc:1467] Processing CreateTablet for tablet a11df4e275f64325987e10de30423ffe (DEFAULT_TABLE table=test-table [id=717f8bac3640407bb2581c4cdc4ebf22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:09.378309 24458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11df4e275f64325987e10de30423ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:09.377979 24600 tablet_service.cc:1467] Processing CreateTablet for tablet a11df4e275f64325987e10de30423ffe (DEFAULT_TABLE table=test-table [id=717f8bac3640407bb2581c4cdc4ebf22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:09.379292 24600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a11df4e275f64325987e10de30423ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:09.387598 24692 tablet_bootstrap.cc:492] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a: Bootstrap starting.
I20260808 23:40:09.391083 24692 tablet_bootstrap.cc:654] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:09.392213 24692 log.cc:824] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:09.394145 24697 tablet_bootstrap.cc:492] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: Bootstrap starting.
I20260808 23:40:09.394836 24695 tablet_bootstrap.cc:492] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: Bootstrap starting.
I20260808 23:40:09.395328 24692 tablet_bootstrap.cc:492] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a: No bootstrap required, opened a new log
I20260808 23:40:09.395630 24692 ts_tablet_manager.cc:1397] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260808 23:40:09.397909 24695 tablet_bootstrap.cc:654] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:09.398344 24697 tablet_bootstrap.cc:654] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:09.398944 24695 log.cc:824] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:09.399540 24697 log.cc:824] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:09.401535 24695 tablet_bootstrap.cc:492] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: No bootstrap required, opened a new log
I20260808 23:40:09.401788 24695 ts_tablet_manager.cc:1397] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260808 23:40:09.404023 24692 raft_consensus.cc:348] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.404381 24692 raft_consensus.cc:374] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:09.404515 24692 raft_consensus.cc:729] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e20ad4509c44bbdb83bccb6e68a206a, State: Initialized, Role: FOLLOWER
I20260808 23:40:09.405128 24697 tablet_bootstrap.cc:492] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: No bootstrap required, opened a new log
I20260808 23:40:09.405121 24692 consensus_queue.cc:260] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.405390 24697 ts_tablet_manager.cc:1397] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260808 23:40:09.407658 24387 heartbeater.cc:503] Master 127.18.186.190:42135 was elected leader, sending a full tablet report...
I20260808 23:40:09.408309 24692 ts_tablet_manager.cc:1428] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260808 23:40:09.412014 24695 raft_consensus.cc:348] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.412446 24695 raft_consensus.cc:374] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:09.412546 24695 raft_consensus.cc:729] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd1773661e84de1a8227de95b60c14a, State: Initialized, Role: FOLLOWER
I20260808 23:40:09.413131 24695 consensus_queue.cc:260] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.414217 24529 heartbeater.cc:503] Master 127.18.186.190:42135 was elected leader, sending a full tablet report...
I20260808 23:40:09.414878 24695 ts_tablet_manager.cc:1428] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260808 23:40:09.415726 24697 raft_consensus.cc:348] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.416023 24697 raft_consensus.cc:374] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:09.416224 24697 raft_consensus.cc:729] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1daf0c025833426f95adc8dadf9c4156, State: Initialized, Role: FOLLOWER
I20260808 23:40:09.416818 24697 consensus_queue.cc:260] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.418162 24671 heartbeater.cc:503] Master 127.18.186.190:42135 was elected leader, sending a full tablet report...
I20260808 23:40:09.418889 24697 ts_tablet_manager.cc:1428] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
W20260808 23:40:09.430298 24388 tablet.cc:2369] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:40:09.493719 24700 raft_consensus.cc:482] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:09.494081 24700 raft_consensus.cc:504] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.495894 24700 leader_election.cc:288] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dd1773661e84de1a8227de95b60c14a (127.18.186.130:36129), 1daf0c025833426f95adc8dadf9c4156 (127.18.186.131:41855)
I20260808 23:40:09.499277 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.496215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:36165 (local address 127.18.186.130:36129)
0808 23:40:09.496433 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.496442 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.497148 (+ 706us) server_negotiation.cc:407] Connection header received
0808 23:40:09.497352 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.497363 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.497423 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.497534 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.498631 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.498636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.498641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.498664 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.498696 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.498707 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.498826 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.498916 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.498986 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:40:09.499483 24704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.496142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:36129 (local address 127.18.186.129:36165)
0808 23:40:09.497019 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.497035 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:09.497200 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:09.497566 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:09.497571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:09.497596 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:09.498356 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:09.498362 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:09.498383 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:09.498455 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:09.498801 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:09.498806 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:09.498810 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:40:09.498949 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 23:40:09.498999 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":557,"threads_started":1}
I20260808 23:40:09.499543 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.496510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:55059 (local address 127.18.186.131:41855)
0808 23:40:09.496724 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.496734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.497418 (+ 684us) server_negotiation.cc:407] Connection header received
0808 23:40:09.497593 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.497599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.497659 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.497768 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.498850 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.498856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.498861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.498880 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.498921 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.498932 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.499073 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.499169 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.499236 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:40:09.499825 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.496407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:41855 (local address 127.18.186.129:55059)
0808 23:40:09.497295 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.497309 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:09.497459 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:09.497791 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:09.497795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:09.497827 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:09.498550 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:09.498552 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:09.498564 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:09.498599 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:09.499040 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:09.499044 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:09.499050 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 23:40:09.499212 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 23:40:09.499262 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":554,"threads_started":1}
I20260808 23:40:09.500527 24480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11df4e275f64325987e10de30423ffe" candidate_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd1773661e84de1a8227de95b60c14a" is_pre_election: true
I20260808 23:40:09.500787 24621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11df4e275f64325987e10de30423ffe" candidate_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daf0c025833426f95adc8dadf9c4156" is_pre_election: true
I20260808 23:40:09.501039 24480 raft_consensus.cc:2454] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e20ad4509c44bbdb83bccb6e68a206a in term 0.
I20260808 23:40:09.501328 24621 raft_consensus.cc:2454] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e20ad4509c44bbdb83bccb6e68a206a in term 0.
I20260808 23:40:09.501830 24266 leader_election.cc:302] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dd1773661e84de1a8227de95b60c14a, 7e20ad4509c44bbdb83bccb6e68a206a; no voters:
I20260808 23:40:09.502288 24700 raft_consensus.cc:2781] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:09.502439 24700 raft_consensus.cc:482] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:09.502550 24700 raft_consensus.cc:3037] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:09.505214 24700 raft_consensus.cc:504] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.505854 24700 leader_election.cc:288] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [CANDIDATE]: Term 1 election: Requested vote from peers 5dd1773661e84de1a8227de95b60c14a (127.18.186.130:36129), 1daf0c025833426f95adc8dadf9c4156 (127.18.186.131:41855)
I20260808 23:40:09.506263 24480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11df4e275f64325987e10de30423ffe" candidate_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd1773661e84de1a8227de95b60c14a"
I20260808 23:40:09.506369 24621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a11df4e275f64325987e10de30423ffe" candidate_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daf0c025833426f95adc8dadf9c4156"
I20260808 23:40:09.506485 24480 raft_consensus.cc:3037] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:09.506623 24621 raft_consensus.cc:3037] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:09.508880 24480 raft_consensus.cc:2454] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e20ad4509c44bbdb83bccb6e68a206a in term 1.
I20260808 23:40:09.508951 24621 raft_consensus.cc:2454] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e20ad4509c44bbdb83bccb6e68a206a in term 1.
I20260808 23:40:09.509327 24266 leader_election.cc:302] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dd1773661e84de1a8227de95b60c14a, 7e20ad4509c44bbdb83bccb6e68a206a; no voters:
I20260808 23:40:09.509630 24700 raft_consensus.cc:2781] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:09.510089 24700 raft_consensus.cc:686] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [term 1 LEADER]: Becoming Leader. State: Replica: 7e20ad4509c44bbdb83bccb6e68a206a, State: Running, Role: LEADER
I20260808 23:40:09.510634 24700 consensus_queue.cc:237] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } }
I20260808 23:40:09.515278 24182 catalog_manager.cc:5495] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e20ad4509c44bbdb83bccb6e68a206a (127.18.186.129). New cstate: current_term: 1 leader_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e20ad4509c44bbdb83bccb6e68a206a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 } health_report { overall_health: UNKNOWN } } }
I20260808 23:40:09.528285 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.526096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.18.186.129:39153)
0808 23:40:09.526271 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.526275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.526656 (+ 381us) server_negotiation.cc:407] Connection header received
0808 23:40:09.526745 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.526749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.526826 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.526918 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.527795 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.527799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.527802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.527817 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.527850 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.527861 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.527969 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.528034 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.528079 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:40:09.536300 24480 raft_consensus.cc:1261] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 1 FOLLOWER]: Refusing update from remote peer 7e20ad4509c44bbdb83bccb6e68a206a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:09.536299 24621 raft_consensus.cc:1261] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 1 FOLLOWER]: Refusing update from remote peer 7e20ad4509c44bbdb83bccb6e68a206a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:09.537279 24707 consensus_queue.cc:1035] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1daf0c025833426f95adc8dadf9c4156" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:09.537608 24700 consensus_queue.cc:1035] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd1773661e84de1a8227de95b60c14a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:09.548588 24710 mvcc.cc:205] Tried to move back new op lower bound from 7316407949450854400 to 7316407949359595520. Current Snapshot: MvccSnapshot[applied={T|T < 7316407949450854400}]
I20260808 23:40:09.549682 24711 mvcc.cc:205] Tried to move back new op lower bound from 7316407949450854400 to 7316407949359595520. Current Snapshot: MvccSnapshot[applied={T|T < 7316407949450854400}]
I20260808 23:40:09.561625 24182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:09.561882 24182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:09.565172 24182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 575e36693d834b48a79dbcab9e8e1ed4: Sending DeleteTablet for 3 replicas of tablet a11df4e275f64325987e10de30423ffe
I20260808 23:40:09.566208 24458 tablet_service.cc:1514] Processing DeleteTablet for tablet a11df4e275f64325987e10de30423ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:09 UTC) from {username='slave'} at 127.0.0.1:42736
I20260808 23:40:09.566257 24600 tablet_service.cc:1514] Processing DeleteTablet for tablet a11df4e275f64325987e10de30423ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:09 UTC) from {username='slave'} at 127.0.0.1:33734
I20260808 23:40:09.566224 24313 tablet_service.cc:1514] Processing DeleteTablet for tablet a11df4e275f64325987e10de30423ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:09 UTC) from {username='slave'} at 127.0.0.1:51572
I20260808 23:40:09.569016 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 24247
I20260808 23:40:09.601991 24689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.570732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.18.186.129:39153)
0808 23:40:09.570968 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.570978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.598372 (+ 27394us) server_negotiation.cc:407] Connection header received
0808 23:40:09.599025 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.599035 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.599127 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.599226 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.600757 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.600770 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.600778 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.600806 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.600855 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.600876 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.601040 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.601420 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.601504 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:40:09.602362 24722 tablet_replica.cc:330] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: stopping tablet replica
I20260808 23:40:09.603525 24720 tablet_replica.cc:330] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: stopping tablet replica
I20260808 23:40:09.604017 24720 raft_consensus.cc:2229] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:09.604525 24720 raft_consensus.cc:2258] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:09.604537 24382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:09.605073 24722 raft_consensus.cc:2229] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:09.605621 24722 raft_consensus.cc:2258] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:09.608345 24720 ts_tablet_manager.cc:1905] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:09.611017 24722 ts_tablet_manager.cc:1905] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:09.615659 24720 ts_tablet_manager.cc:1918] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:09.615839 24720 log.cc:1196] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/a11df4e275f64325987e10de30423ffe
I20260808 23:40:09.616218 24722 ts_tablet_manager.cc:1918] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:09.616384 24722 log.cc:1196] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/a11df4e275f64325987e10de30423ffe
I20260808 23:40:09.616405 24720 ts_tablet_manager.cc:1939] T a11df4e275f64325987e10de30423ffe P 5dd1773661e84de1a8227de95b60c14a: Deleting consensus metadata
I20260808 23:40:09.616859 24722 ts_tablet_manager.cc:1939] T a11df4e275f64325987e10de30423ffe P 1daf0c025833426f95adc8dadf9c4156: Deleting consensus metadata
I20260808 23:40:09.618355 24167 catalog_manager.cc:4841] TS 5dd1773661e84de1a8227de95b60c14a (127.18.186.130:36129): tablet a11df4e275f64325987e10de30423ffe (table test-table [id=717f8bac3640407bb2581c4cdc4ebf22]) successfully deleted
I20260808 23:40:09.618752 24166 catalog_manager.cc:4841] TS 1daf0c025833426f95adc8dadf9c4156 (127.18.186.131:41855): tablet a11df4e275f64325987e10de30423ffe (table test-table [id=717f8bac3640407bb2581c4cdc4ebf22]) successfully deleted
I20260808 23:40:09.703323 24721 tablet_replica.cc:330] T a11df4e275f64325987e10de30423ffe P 7e20ad4509c44bbdb83bccb6e68a206a: stopping tablet replica
I20260808 23:40:09.703634 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 24247
I20260808 23:40:09.724232 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 24392
W20260808 23:40:09.724182 24168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.186.129:39153 (error 108)
I20260808 23:40:09.726179 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.725755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39153 (local address 127.0.0.1:51592)
0808 23:40:09.725934 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.726049 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:39153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 23:40:09.726507 24168 catalog_manager.cc:4584] TS 7e20ad4509c44bbdb83bccb6e68a206a (127.18.186.129:39153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a11df4e275f64325987e10de30423ffe: Network error: Client connection negotiation failed: client connection to 127.18.186.129:39153: connect: Connection refused (error 111)
I20260808 23:40:09.727672 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.725191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.18.186.130:36129)
0808 23:40:09.725399 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.725405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.725522 (+ 117us) server_negotiation.cc:407] Connection header received
0808 23:40:09.725732 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.725738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.725841 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.725954 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.726995 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.727001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.727006 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.727025 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.727066 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.727078 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.727212 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.727304 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.727365 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:40:09.728353 24523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:09.780254 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.779785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39153 (local address 127.0.0.1:51598)
0808 23:40:09.780000 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.780085 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:39153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 23:40:09.781342 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.781025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39153 (local address 127.0.0.1:51614)
0808 23:40:09.781155 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.781235 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:39153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 23:40:09.815682 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 24392
I20260808 23:40:09.836736 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 24532
I20260808 23:40:09.839587 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.837575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.18.186.131:41855)
0808 23:40:09.837759 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:09.837764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:09.837784 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:40:09.837941 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:09.837946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:09.838023 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:09.838106 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:09.838941 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:09.838946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:09.838949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:09.838963 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:09.838995 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:09.839010 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:09.839115 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:09.839184 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 23:40:09.839229 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:40:09.840243 24665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:09.842764 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.842277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39153 (local address 127.0.0.1:51626)
0808 23:40:09.842472 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.842596 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:39153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 23:40:09.844121 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.843730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39153 (local address 127.0.0.1:51636)
0808 23:40:09.843928 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.844008 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:39153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 23:40:09.924961 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.924431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39153 (local address 127.0.0.1:51642)
0808 23:40:09.924677 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.924796 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:39153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 23:40:09.926056 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:09.925653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39153 (local address 127.0.0.1:51650)
0808 23:40:09.925863 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:09.925943 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:39153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260808 23:40:09.940791 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 24532
I20260808 23:40:09.959875 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 24150
I20260808 23:40:09.961097 24215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:10.069489 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 24150
2026-08-08T23:40:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2669 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 23:40:10.113991 19178 mini_oidc.cc:152] Starting JWKS server
I20260808 23:40:10.117024 19178 webserver.cc:460] Webserver started at https://127.0.0.1:35437/ using document root <none> and password file <none>
I20260808 23:40:10.117393 19178 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:40:10.118140 19178 webserver.cc:460] Webserver started at http://127.0.0.1:42359/ using document root <none> and password file <none>
2026-08-08T23:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:10Z Disabled control of system clock
I20260808 23:40:10.151530 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:41519
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:41519
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35437/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/testchainca.cert with env {}
W20260808 23:40:10.476927 24734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:10.477278 24734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:10.477450 24734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:10.484562 24734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:10.484716 24734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:10.484814 24734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:10.484932 24734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:10.485023 24734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:40:10.485121 24734 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:40:10.485198 24734 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35437/jwks.json
I20260808 23:40:10.495064 24734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:41519
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:41519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35437/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:10.496575 24734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:10.497927 24734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:10.510160 24740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:10.510238 24739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:10.510215 24742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:10.511049 24734 server_base.cc:1034] running on GCE node
I20260808 23:40:10.511919 24734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:10.513322 24734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:10.514572 24734 hybrid_clock.cc:648] HybridClock initialized: now 1786232410514528 us; error 58 us; skew 500 ppm
I20260808 23:40:10.514936 24734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:10.523207 24734 webserver.cc:460] Webserver started at http://127.18.186.190:43057/ using document root <none> and password file <none>
I20260808 23:40:10.523986 24734 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:10.524127 24734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:10.524453 24734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:10.527375 24734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "45f8eab9ecc041daba758118c5bc4d4e"
format_stamp: "Formatted at 2026-08-08 23:40:10 on dist-test-slave-5k54"
server_key: "e5187c82f7aa605cc85359e7347d31e1"
server_key_iv: "8e2fcff0999d4cb1f0cd3306b411e459"
server_key_version: "encryptionkey@0"
I20260808 23:40:10.528275 24734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "45f8eab9ecc041daba758118c5bc4d4e"
format_stamp: "Formatted at 2026-08-08 23:40:10 on dist-test-slave-5k54"
server_key: "e5187c82f7aa605cc85359e7347d31e1"
server_key_iv: "8e2fcff0999d4cb1f0cd3306b411e459"
server_key_version: "encryptionkey@0"
I20260808 23:40:10.533463 24734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 23:40:10.536942 24748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:10.538292 24734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:40:10.538481 24734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "45f8eab9ecc041daba758118c5bc4d4e"
format_stamp: "Formatted at 2026-08-08 23:40:10 on dist-test-slave-5k54"
server_key: "e5187c82f7aa605cc85359e7347d31e1"
server_key_iv: "8e2fcff0999d4cb1f0cd3306b411e459"
server_key_version: "encryptionkey@0"
I20260808 23:40:10.538709 24734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:10.560119 24734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:10.561909 24734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:10.562330 24734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:10.581295 24734 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:41519
I20260808 23:40:10.581334 24802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:41519 every 8 connection(s)
I20260808 23:40:10.582679 24734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:10.587647 24803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:10.592651 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 24734
I20260808 23:40:10.592893 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:10.593528 19178 external_mini_cluster.cc:1428] Setting key cf3256a8dd804a76e27973cd1e571bcb
I20260808 23:40:10.601051 24803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e: Bootstrap starting.
I20260808 23:40:10.605028 24803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:10.606465 24803 log.cc:824] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:10.610023 24803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e: No bootstrap required, opened a new log
I20260808 23:40:10.610312 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:10.595469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.18.186.190:41519)
0808 23:40:10.596700 (+ 1231us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:10.596720 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:10.596777 (+ 57us) server_negotiation.cc:407] Connection header received
0808 23:40:10.599528 (+ 2751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:10.599559 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:10.600159 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:10.600630 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:10.601202 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:10.603520 (+ 2318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:10.604854 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:10.606005 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:10.606230 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:10.607391 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:10.607398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:10.607412 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:10.608163 (+ 751us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:10.608274 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:10.608301 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:10.608430 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:10.608650 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 23:40:10.608966 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":357,"threads_started":1}
I20260808 23:40:10.614041 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:40:10.619029 24803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f8eab9ecc041daba758118c5bc4d4e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41519 } }
I20260808 23:40:10.619408 24803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:10.619556 24803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45f8eab9ecc041daba758118c5bc4d4e, State: Initialized, Role: FOLLOWER
I20260808 23:40:10.620283 24803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f8eab9ecc041daba758118c5bc4d4e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41519 } }
I20260808 23:40:10.620579 24803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:10.620725 24803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:10.620937 24803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:10.623288 24803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f8eab9ecc041daba758118c5bc4d4e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41519 } }
I20260808 23:40:10.623855 24803 leader_election.cc:302] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45f8eab9ecc041daba758118c5bc4d4e; no voters:
I20260808 23:40:10.624467 24803 leader_election.cc:288] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:10.624636 24809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:10.625406 24809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [term 1 LEADER]: Becoming Leader. State: Replica: 45f8eab9ecc041daba758118c5bc4d4e, State: Running, Role: LEADER
I20260808 23:40:10.625946 24809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f8eab9ecc041daba758118c5bc4d4e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41519 } }
I20260808 23:40:10.626679 24803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:10.628836 24811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45f8eab9ecc041daba758118c5bc4d4e. Latest consensus state: current_term: 1 leader_uuid: "45f8eab9ecc041daba758118c5bc4d4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f8eab9ecc041daba758118c5bc4d4e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41519 } } }
I20260808 23:40:10.629226 24811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:10.630560 24810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45f8eab9ecc041daba758118c5bc4d4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45f8eab9ecc041daba758118c5bc4d4e" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41519 } } }
I20260808 23:40:10.630955 24810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:10.632278 24814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:10.637789 24814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:10.647820 24814 catalog_manager.cc:1347] Generated new cluster ID: c39fe008b793457cae080cecf733352b
I20260808 23:40:10.647959 24814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:10.668752 24814 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:10.670653 24814 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:10.678202 24814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45f8eab9ecc041daba758118c5bc4d4e: Generated new TSK 0
I20260808 23:40:10.679369 24814 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 23:40:10.719803 24835 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 23:40:10.720088 24835 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 23:40:10.721251 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:10.698699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.18.186.190:41519)
0808 23:40:10.698913 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:10.698920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:10.699375 (+ 455us) server_negotiation.cc:407] Connection header received
0808 23:40:10.699766 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:10.699772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:10.699868 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:10.700014 (+ 146us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:40:10.700552 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:10.701676 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:10.703083 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:10.703853 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:10.703975 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:10.704503 (+ 528us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:40:10.720564 (+ 16061us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 23:40:10.720940 (+ 376us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36862: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 23:40:10.721393 24806 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36862: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 23:40:10.722851 24833 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:10.698567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41519 (local address 127.0.0.1:36862)
0808 23:40:10.699277 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:10.699293 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:10.699484 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:10.700057 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:10.700061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:10.700071 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:40:10.700393 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:10.700400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:10.701809 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:10.701813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:10.702948 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:10.702959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:10.704043 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:10.704048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:10.704169 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:10.704226 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:40:10.720869 (+ 16643us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 23:40:10.721152 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.186.190:41519: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":377,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.186.190:41519: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35437/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 23:40:10.724717 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 24734
I20260808 23:40:10.725942 24798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:10.819120 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 24734
2026-08-08T23:40:10Z chronyd exiting
I20260808 23:40:10.874291 19178 test_util.cc:175] -----------------------------------------------
I20260808 23:40:10.874477 19178 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786232366675349-19178-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (765 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 23:40:10.879519 19178 mini_oidc.cc:152] Starting JWKS server
I20260808 23:40:10.881353 19178 webserver.cc:460] Webserver started at https://127.0.0.1:42679/ using document root <none> and password file <none>
I20260808 23:40:10.881538 19178 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:40:10.882169 19178 webserver.cc:460] Webserver started at http://127.0.0.1:36815/ using document root <none> and password file <none>
2026-08-08T23:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:10Z Disabled control of system clock
I20260808 23:40:10.913713 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:38503
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:38503
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42679/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/testchainca.cert with env {}
W20260808 23:40:11.234215 24842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:11.234504 24842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:11.234592 24842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:11.241679 24842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:11.241787 24842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:11.241827 24842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:11.241890 24842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:11.241930 24842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:40:11.241991 24842 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:40:11.242031 24842 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42679/jwks.json
I20260808 23:40:11.252152 24842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:38503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:38503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42679/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:11.253458 24842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:11.254734 24842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:11.266908 24848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:11.266924 24849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:11.267004 24851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:11.267882 24842 server_base.cc:1034] running on GCE node
I20260808 23:40:11.268610 24842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:11.269987 24842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:11.271212 24842 hybrid_clock.cc:648] HybridClock initialized: now 1786232411271178 us; error 48 us; skew 500 ppm
I20260808 23:40:11.271616 24842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:11.273726 24842 webserver.cc:460] Webserver started at http://127.18.186.190:36597/ using document root <none> and password file <none>
I20260808 23:40:11.274403 24842 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:11.274549 24842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:11.274863 24842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:11.277635 24842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "4c5a41eb35274feeaf232b45d33f17ac"
format_stamp: "Formatted at 2026-08-08 23:40:11 on dist-test-slave-5k54"
server_key: "d0fb6c6e62b5c5fbbd7f364e4c0f94cd"
server_key_iv: "2212b38afb4f7302428b3b6e44243da6"
server_key_version: "encryptionkey@0"
I20260808 23:40:11.278429 24842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "4c5a41eb35274feeaf232b45d33f17ac"
format_stamp: "Formatted at 2026-08-08 23:40:11 on dist-test-slave-5k54"
server_key: "d0fb6c6e62b5c5fbbd7f364e4c0f94cd"
server_key_iv: "2212b38afb4f7302428b3b6e44243da6"
server_key_version: "encryptionkey@0"
I20260808 23:40:11.283507 24842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 23:40:11.286809 24857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:11.288125 24842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:11.288302 24842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "4c5a41eb35274feeaf232b45d33f17ac"
format_stamp: "Formatted at 2026-08-08 23:40:11 on dist-test-slave-5k54"
server_key: "d0fb6c6e62b5c5fbbd7f364e4c0f94cd"
server_key_iv: "2212b38afb4f7302428b3b6e44243da6"
server_key_version: "encryptionkey@0"
I20260808 23:40:11.288518 24842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:11.310312 24842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:11.311095 24842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:11.311401 24842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:11.329406 24842 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:38503
I20260808 23:40:11.329416 24911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:38503 every 8 connection(s)
I20260808 23:40:11.330873 24842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:11.333240 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 24842
I20260808 23:40:11.333436 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:11.334085 19178 external_mini_cluster.cc:1428] Setting key fad14644489fefd197551c646625bee7
I20260808 23:40:11.334852 24912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:11.348223 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:11.336091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.18.186.190:38503)
0808 23:40:11.336997 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:11.337008 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:11.337046 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:40:11.339310 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:11.339331 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:11.339855 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:11.340166 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:11.340795 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:11.342142 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:11.343936 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:11.344589 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:11.344726 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:11.346002 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:11.346010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:11.346025 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:11.346553 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:11.346627 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:11.346651 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:11.346731 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:11.346964 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 23:40:11.347283 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":333,"threads_started":1}
I20260808 23:40:11.348506 24912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac: Bootstrap starting.
I20260808 23:40:11.351658 24912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:11.351825 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:40:11.352860 24912 log.cc:824] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:11.355679 24912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac: No bootstrap required, opened a new log
I20260808 23:40:11.362921 24912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5a41eb35274feeaf232b45d33f17ac" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38503 } }
I20260808 23:40:11.363314 24912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:11.363473 24912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c5a41eb35274feeaf232b45d33f17ac, State: Initialized, Role: FOLLOWER
I20260808 23:40:11.364082 24912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5a41eb35274feeaf232b45d33f17ac" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38503 } }
I20260808 23:40:11.364326 24912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:11.364485 24912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:11.364657 24912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:11.366748 24912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5a41eb35274feeaf232b45d33f17ac" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38503 } }
I20260808 23:40:11.367276 24912 leader_election.cc:302] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c5a41eb35274feeaf232b45d33f17ac; no voters:
I20260808 23:40:11.367848 24912 leader_election.cc:288] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:11.367996 24917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:11.369287 24917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [term 1 LEADER]: Becoming Leader. State: Replica: 4c5a41eb35274feeaf232b45d33f17ac, State: Running, Role: LEADER
I20260808 23:40:11.369822 24917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5a41eb35274feeaf232b45d33f17ac" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38503 } }
I20260808 23:40:11.370102 24912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:11.372584 24920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c5a41eb35274feeaf232b45d33f17ac. Latest consensus state: current_term: 1 leader_uuid: "4c5a41eb35274feeaf232b45d33f17ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5a41eb35274feeaf232b45d33f17ac" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38503 } } }
I20260808 23:40:11.373124 24920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:11.372594 24918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c5a41eb35274feeaf232b45d33f17ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c5a41eb35274feeaf232b45d33f17ac" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38503 } } }
I20260808 23:40:11.373648 24918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:11.374089 24926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:11.380421 24926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:11.394567 24926 catalog_manager.cc:1347] Generated new cluster ID: 539a3d6bb96b458f8a3fcdcf08fdc0c0
I20260808 23:40:11.394742 24926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:11.416838 24926 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:11.418612 24926 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:11.425711 24926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c5a41eb35274feeaf232b45d33f17ac: Generated new TSK 0
I20260808 23:40:11.426509 24926 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 23:40:11.466981 24944 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 23:40:11.467114 24944 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 23:40:11.468297 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:11.441257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57598 (local address 127.18.186.190:38503)
0808 23:40:11.441455 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:11.441462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:11.441829 (+ 367us) server_negotiation.cc:407] Connection header received
0808 23:40:11.442081 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:11.442085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:11.442149 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:11.442293 (+ 144us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:40:11.442794 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:11.443606 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:11.445001 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:11.445477 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:11.445611 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:11.446105 (+ 494us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:40:11.467538 (+ 21433us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 23:40:11.467949 (+ 411us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57598: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 23:40:11.468437 24915 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57598: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 23:40:11.468801 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:11.441159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:38503 (local address 127.0.0.1:57598)
0808 23:40:11.441726 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:11.441741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:11.441905 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:11.442311 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:11.442316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:11.442328 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:40:11.442617 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:11.442626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:11.443747 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:11.443758 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:11.444853 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:11.444864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:11.445657 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:11.445666 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:11.445813 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:11.445845 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:40:11.467781 (+ 21936us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 23:40:11.468055 (+ 274us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.186.190:38503: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":325,"threads_started":1}
I20260808 23:40:11.470118 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 24842
I20260808 23:40:11.471470 24907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:11.565526 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 24842
2026-08-08T23:40:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (745 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 23:40:11.623699 19178 mini_oidc.cc:152] Starting JWKS server
I20260808 23:40:11.625630 19178 webserver.cc:460] Webserver started at https://127.0.0.1:34309/ using document root <none> and password file <none>
I20260808 23:40:11.625900 19178 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:40:11.626551 19178 webserver.cc:460] Webserver started at http://127.0.0.1:43529/ using document root <none> and password file <none>
2026-08-08T23:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:11Z Disabled control of system clock
I20260808 23:40:11.657938 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:36493
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:33991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:36493
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34309/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 23:40:11.980532 24951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:11.980876 24951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:11.981045 24951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:11.988183 24951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:11.988322 24951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:11.988420 24951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:11.988511 24951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:11.988579 24951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:40:11.988665 24951 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:40:11.988725 24951 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34309/jwks.json
I20260808 23:40:11.997664 24951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:36493
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:36493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34309/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:11.998997 24951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:12.000918 24951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:12.013100 24957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:12.013128 24958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:12.013324 24960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:12.014207 24951 server_base.cc:1034] running on GCE node
I20260808 23:40:12.014945 24951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:12.016338 24951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:12.017572 24951 hybrid_clock.cc:648] HybridClock initialized: now 1786232412017530 us; error 52 us; skew 500 ppm
I20260808 23:40:12.017938 24951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:12.020283 24951 webserver.cc:460] Webserver started at http://127.18.186.190:45089/ using document root <none> and password file <none>
I20260808 23:40:12.020987 24951 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:12.021119 24951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:12.021450 24951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:12.024554 24951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "05326d34a3724806809890c2b451dc41"
format_stamp: "Formatted at 2026-08-08 23:40:12 on dist-test-slave-5k54"
server_key: "6d027657d0cb9c781ce03571682f7fde"
server_key_iv: "f9746c2633e0da68d5ba8a7dec54b92e"
server_key_version: "encryptionkey@0"
I20260808 23:40:12.025367 24951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "05326d34a3724806809890c2b451dc41"
format_stamp: "Formatted at 2026-08-08 23:40:12 on dist-test-slave-5k54"
server_key: "6d027657d0cb9c781ce03571682f7fde"
server_key_iv: "f9746c2633e0da68d5ba8a7dec54b92e"
server_key_version: "encryptionkey@0"
I20260808 23:40:12.030447 24951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:40:12.033861 24966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:12.035111 24951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:40:12.035286 24951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "05326d34a3724806809890c2b451dc41"
format_stamp: "Formatted at 2026-08-08 23:40:12 on dist-test-slave-5k54"
server_key: "6d027657d0cb9c781ce03571682f7fde"
server_key_iv: "f9746c2633e0da68d5ba8a7dec54b92e"
server_key_version: "encryptionkey@0"
I20260808 23:40:12.035524 24951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:12.079546 24951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:12.080458 24951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:12.080799 24951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:12.099310 24951 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:36493
I20260808 23:40:12.099349 25020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:36493 every 8 connection(s)
I20260808 23:40:12.100674 24951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:12.104382 25021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:12.107847 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 24951
I20260808 23:40:12.108083 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:12.108912 19178 external_mini_cluster.cc:1428] Setting key 47285c7dfae1b65236ca1f5b420555f4
I20260808 23:40:12.117323 25021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41: Bootstrap starting.
I20260808 23:40:12.120499 25021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:12.121685 25021 log.cc:824] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:12.122175 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:12.110908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.18.186.190:36493)
0808 23:40:12.111930 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:12.111940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:12.111981 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:40:12.113585 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:12.113604 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:12.114059 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:12.114455 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:12.115015 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.116391 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.117806 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.118486 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.118716 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:12.119863 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:12.119871 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:12.119884 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:12.120401 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:12.120494 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:12.120513 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:12.120634 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:12.120873 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 23:40:12.121164 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":334,"threads_started":1}
I20260808 23:40:12.124882 25021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41: No bootstrap required, opened a new log
I20260808 23:40:12.125945 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:40:12.132860 25021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05326d34a3724806809890c2b451dc41" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36493 } }
I20260808 23:40:12.133271 25021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:12.133391 25021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05326d34a3724806809890c2b451dc41, State: Initialized, Role: FOLLOWER
I20260808 23:40:12.133955 25021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05326d34a3724806809890c2b451dc41" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36493 } }
I20260808 23:40:12.134193 25021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:12.134343 25021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:12.134507 25021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:12.136824 25021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05326d34a3724806809890c2b451dc41" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36493 } }
I20260808 23:40:12.137346 25021 leader_election.cc:302] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05326d34a3724806809890c2b451dc41; no voters:
I20260808 23:40:12.137943 25021 leader_election.cc:288] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:12.138070 25028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:12.138906 25028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [term 1 LEADER]: Becoming Leader. State: Replica: 05326d34a3724806809890c2b451dc41, State: Running, Role: LEADER
I20260808 23:40:12.139412 25028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05326d34a3724806809890c2b451dc41" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36493 } }
I20260808 23:40:12.140264 25021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:12.142385 25030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05326d34a3724806809890c2b451dc41. Latest consensus state: current_term: 1 leader_uuid: "05326d34a3724806809890c2b451dc41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05326d34a3724806809890c2b451dc41" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36493 } } }
I20260808 23:40:12.142822 25030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:12.144865 25033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:12.144750 25029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05326d34a3724806809890c2b451dc41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05326d34a3724806809890c2b451dc41" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36493 } } }
I20260808 23:40:12.145169 25029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:12.150612 25033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:12.159823 25033 catalog_manager.cc:1347] Generated new cluster ID: 3a448aaff3604b95bc1f63e0b229850a
I20260808 23:40:12.159934 25033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:12.176721 25033 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:12.177947 25033 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:12.184054 25033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05326d34a3724806809890c2b451dc41: Generated new TSK 0
I20260808 23:40:12.184837 25033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:40:12.208016 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:12.199354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33612 (local address 127.18.186.190:36493)
0808 23:40:12.199568 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:12.199574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:12.199997 (+ 423us) server_negotiation.cc:407] Connection header received
0808 23:40:12.200239 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:12.200245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:12.200360 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:12.200531 (+ 171us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:40:12.200952 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.202074 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.203388 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.204099 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.204236 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:12.204754 (+ 518us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:40:12.207322 (+ 2568us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 23:40:12.207690 (+ 368us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33612: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 23:40:12.208148 25024 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33612: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 23:40:12.210043 25053 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:12.199202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36493 (local address 127.0.0.1:33612)
0808 23:40:12.199890 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:12.199912 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:12.200029 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:12.200545 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:12.200548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:12.200557 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:40:12.200813 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:12.200820 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.202207 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.202211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:12.203241 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:12.203252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.204287 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.204292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:12.204435 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:12.204474 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:40:12.207567 (+ 3093us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 23:40:12.209622 (+ 2055us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.186.190:36493: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34309/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":1744,"thread_start_us":387,"threads_started":1}
I20260808 23:40:12.211125 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 24951
I20260808 23:40:12.212423 25016 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 23:40:12.224167 25054 webserver.cc:533] Webserver: sslize failed:
E20260808 23:40:12.224325 25054 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 23:40:12.305424 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 24951
2026-08-08T23:40:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (739 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T23:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:12Z Disabled control of system clock
I20260808 23:40:12.394270 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:43143
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:42179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:43143
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/testchainca.cert with env {}
W20260808 23:40:12.726305 25059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:12.726591 25059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:12.726754 25059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:12.733906 25059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:12.734005 25059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:12.734107 25059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:12.734217 25059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:12.734299 25059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:40:12.734393 25059 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:40:12.734449 25059 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 23:40:12.744266 25059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:42179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:43143
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:43143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:12.745641 25059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:12.746986 25059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:12.759023 25066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:12.759223 25065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:12.759253 25068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:12.760208 25059 server_base.cc:1034] running on GCE node
I20260808 23:40:12.760962 25059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:12.762315 25059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:12.763520 25059 hybrid_clock.cc:648] HybridClock initialized: now 1786232412763477 us; error 56 us; skew 500 ppm
I20260808 23:40:12.763870 25059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:12.766088 25059 webserver.cc:460] Webserver started at http://127.18.186.190:43325/ using document root <none> and password file <none>
I20260808 23:40:12.766764 25059 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:12.766909 25059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:12.767220 25059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:12.770083 25059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "7e603f6cc2814b098f14c49440be65c7"
format_stamp: "Formatted at 2026-08-08 23:40:12 on dist-test-slave-5k54"
server_key: "28e4ffd8783a43fac3f39a802c38aa90"
server_key_iv: "799557cac73164ba5536dd3c03bb8e38"
server_key_version: "encryptionkey@0"
I20260808 23:40:12.770874 25059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "7e603f6cc2814b098f14c49440be65c7"
format_stamp: "Formatted at 2026-08-08 23:40:12 on dist-test-slave-5k54"
server_key: "28e4ffd8783a43fac3f39a802c38aa90"
server_key_iv: "799557cac73164ba5536dd3c03bb8e38"
server_key_version: "encryptionkey@0"
I20260808 23:40:12.775852 25059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 23:40:12.779130 25074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:12.780449 25059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:12.780649 25059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "7e603f6cc2814b098f14c49440be65c7"
format_stamp: "Formatted at 2026-08-08 23:40:12 on dist-test-slave-5k54"
server_key: "28e4ffd8783a43fac3f39a802c38aa90"
server_key_iv: "799557cac73164ba5536dd3c03bb8e38"
server_key_version: "encryptionkey@0"
I20260808 23:40:12.780869 25059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:12.791082 25059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:12.791870 25059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:12.792176 25059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:12.810245 25059 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:43143
I20260808 23:40:12.810263 25128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:43143 every 8 connection(s)
I20260808 23:40:12.811551 25059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:12.814047 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 25059
I20260808 23:40:12.814265 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:12.814920 19178 external_mini_cluster.cc:1428] Setting key 02ced5f2521069d0e9d9b0aa061280ba
I20260808 23:40:12.815294 25129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:12.830307 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:12.816863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.18.186.190:43143)
0808 23:40:12.817942 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:12.817956 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:12.818011 (+ 55us) server_negotiation.cc:407] Connection header received
0808 23:40:12.820181 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:12.820207 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:12.820806 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:12.821232 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:12.821763 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.823793 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.825183 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.826203 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.826388 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:12.827541 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:12.827552 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:12.827575 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:12.828301 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:12.828392 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:12.828428 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:12.828543 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:12.828732 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 23:40:12.829048 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":342,"threads_started":1}
I20260808 23:40:12.830381 25129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7: Bootstrap starting.
I20260808 23:40:12.833638 25129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:12.834074 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:40:12.834755 25129 log.cc:824] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:12.838321 25129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7: No bootstrap required, opened a new log
I20260808 23:40:12.846068 25129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e603f6cc2814b098f14c49440be65c7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43143 } }
I20260808 23:40:12.846444 25129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:12.846549 25129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e603f6cc2814b098f14c49440be65c7, State: Initialized, Role: FOLLOWER
I20260808 23:40:12.847180 25129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e603f6cc2814b098f14c49440be65c7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43143 } }
I20260808 23:40:12.847414 25129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:12.847612 25129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:12.847786 25129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:12.849972 25129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e603f6cc2814b098f14c49440be65c7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43143 } }
I20260808 23:40:12.850461 25129 leader_election.cc:302] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e603f6cc2814b098f14c49440be65c7; no voters:
I20260808 23:40:12.851038 25129 leader_election.cc:288] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:12.851177 25134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:12.851936 25134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [term 1 LEADER]: Becoming Leader. State: Replica: 7e603f6cc2814b098f14c49440be65c7, State: Running, Role: LEADER
I20260808 23:40:12.852492 25134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e603f6cc2814b098f14c49440be65c7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43143 } }
I20260808 23:40:12.853158 25129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:12.856174 25136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e603f6cc2814b098f14c49440be65c7. Latest consensus state: current_term: 1 leader_uuid: "7e603f6cc2814b098f14c49440be65c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e603f6cc2814b098f14c49440be65c7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43143 } } }
I20260808 23:40:12.856151 25135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e603f6cc2814b098f14c49440be65c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e603f6cc2814b098f14c49440be65c7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43143 } } }
I20260808 23:40:12.856534 25136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:12.856561 25135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:12.858525 25141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:12.863758 25141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:12.876458 25141 catalog_manager.cc:1347] Generated new cluster ID: 59d3e45ebebb40ea85a799a8ab1cbc5b
I20260808 23:40:12.876595 25141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:12.893055 25141 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:12.894366 25141 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:12.902577 25141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e603f6cc2814b098f14c49440be65c7: Generated new TSK 0
I20260808 23:40:12.903367 25141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:40:12.992587 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:12.922004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.18.186.190:43143)
0808 23:40:12.922195 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:12.922201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:12.922740 (+ 539us) server_negotiation.cc:407] Connection header received
0808 23:40:12.922925 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:12.922931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:12.923018 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:12.923173 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:40:12.923644 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.924804 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.926206 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.926912 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.927055 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:12.927551 (+ 496us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:40:12.991803 (+ 64252us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 23:40:12.992254 (+ 451us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48588: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 23:40:12.992704 25159 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:12.922014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43143 (local address 127.0.0.1:48588)
0808 23:40:12.922639 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:12.922658 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:12.922766 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:12.923186 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:12.923189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:12.923197 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:40:12.923499 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:12.923508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.924969 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.924976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:12.926052 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:12.926063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:12.927070 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:12.927074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:12.927223 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:12.927252 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:40:12.992119 (+ 64867us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 23:40:12.992377 (+ 258us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.186.190:43143: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":340,"threads_started":1}
W20260808 23:40:12.992735 25132 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48588: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260808 23:40:12.994762 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 25059
I20260808 23:40:12.995999 25124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:13.095480 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 25059
2026-08-08T23:40:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (770 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (414 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (415 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:40:14 dist-test-slave-5k54 krb5kdc[25175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:40:14 dist-test-slave-5k54 krb5kdc[25175](info): set up 2 sockets
Aug 08 23:40:14 dist-test-slave-5k54 krb5kdc[25175](info): commencing operation
krb5kdc: starting...
W20260808 23:40:15.067768 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:40:15 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:40:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:40:15.298812 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:39953
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:39953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:15.622258 25189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:15.622565 25189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:15.622718 25189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:15.630057 25189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:15.630167 25189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:15.630270 25189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:15.630443 25189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:15.630513 25189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:15.640625 25189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:39953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:39953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:15.642270 25189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:15.643834 25189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:15.655857 25194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:15.655818 25197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:15.655887 25195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:15.656730 25189 server_base.cc:1034] running on GCE node
I20260808 23:40:15.657605 25189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:15.658974 25189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:15.660212 25189 hybrid_clock.cc:648] HybridClock initialized: now 1786232415660161 us; error 68 us; skew 500 ppm
I20260808 23:40:15.660591 25189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:15 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232415, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:15.666633 25189 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:40:15.668179 25189 webserver.cc:460] Webserver started at http://127.18.186.190:34415/ using document root <none> and password file <none>
I20260808 23:40:15.668884 25189 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:15.669036 25189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:15.669361 25189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:15.672452 25189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "f817cd56237247ec8040a00019827479"
format_stamp: "Formatted at 2026-08-08 23:40:15 on dist-test-slave-5k54"
server_key: "445277272ab654e4d84d1cb0eb2917de"
server_key_iv: "4a55ed21be8a90d2f0880464ced57911"
server_key_version: "encryptionkey@0"
I20260808 23:40:15.673241 25189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "f817cd56237247ec8040a00019827479"
format_stamp: "Formatted at 2026-08-08 23:40:15 on dist-test-slave-5k54"
server_key: "445277272ab654e4d84d1cb0eb2917de"
server_key_iv: "4a55ed21be8a90d2f0880464ced57911"
server_key_version: "encryptionkey@0"
I20260808 23:40:15.678665 25189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:40:15.682031 25206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:15.683297 25189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:40:15.683521 25189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "f817cd56237247ec8040a00019827479"
format_stamp: "Formatted at 2026-08-08 23:40:15 on dist-test-slave-5k54"
server_key: "445277272ab654e4d84d1cb0eb2917de"
server_key_iv: "4a55ed21be8a90d2f0880464ced57911"
server_key_version: "encryptionkey@0"
I20260808 23:40:15.683729 25189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:15.708645 25189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:15.713055 25189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:15.713353 25189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:15.731355 25189 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:39953
I20260808 23:40:15.731392 25260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:39953 every 8 connection(s)
I20260808 23:40:15.732713 25189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:15.736301 25261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:15.738958 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 25189
I20260808 23:40:15.739184 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:15.740036 19178 external_mini_cluster.cc:1428] Setting key 6e785d0d009c7ecef267369ac1033df4
I20260808 23:40:15.748339 25261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479: Bootstrap starting.
I20260808 23:40:15.751212 25261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:15.752203 25261 log.cc:824] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:40:15 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:15.755189 25261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479: No bootstrap required, opened a new log
I20260808 23:40:15.762794 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:15.742258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.18.186.190:39953)
0808 23:40:15.743043 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:15.743054 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:15.743093 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:40:15.744857 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:15.744878 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:15.745250 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:15.745641 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:15.747120 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:15.748550 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:15.749868 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:15.750532 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:15.750720 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:15.755381 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:15.755391 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:15.755404 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:15.755478 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:15.758694 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:15.759399 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:15.759406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:15.759411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:15.759630 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:15.760034 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:15.760045 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:15.760050 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:15.760578 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:15.760893 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:15.761416 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 23:40:15.761837 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":347,"threads_started":1}
I20260808 23:40:15.762797 25261 raft_consensus.cc:348] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f817cd56237247ec8040a00019827479" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 39953 } }
I20260808 23:40:15.763360 25261 raft_consensus.cc:374] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:15.763504 25261 raft_consensus.cc:729] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f817cd56237247ec8040a00019827479, State: Initialized, Role: FOLLOWER
I20260808 23:40:15.764142 25261 consensus_queue.cc:260] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f817cd56237247ec8040a00019827479" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 39953 } }
I20260808 23:40:15.764436 25261 raft_consensus.cc:388] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:15.764585 25261 raft_consensus.cc:482] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:15.764781 25261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:15.767156 25261 raft_consensus.cc:504] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f817cd56237247ec8040a00019827479" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 39953 } }
I20260808 23:40:15.767700 25261 leader_election.cc:302] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f817cd56237247ec8040a00019827479; no voters:
I20260808 23:40:15.768260 25261 leader_election.cc:288] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:15.768416 25266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:15.769136 25266 raft_consensus.cc:686] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [term 1 LEADER]: Becoming Leader. State: Replica: f817cd56237247ec8040a00019827479, State: Running, Role: LEADER
I20260808 23:40:15.769649 25266 consensus_queue.cc:237] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f817cd56237247ec8040a00019827479" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 39953 } }
I20260808 23:40:15.770287 25261 sys_catalog.cc:564] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:15.772332 25267 sys_catalog.cc:455] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f817cd56237247ec8040a00019827479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f817cd56237247ec8040a00019827479" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 39953 } } }
I20260808 23:40:15.772706 25267 sys_catalog.cc:458] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:15.774758 25268 sys_catalog.cc:455] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f817cd56237247ec8040a00019827479. Latest consensus state: current_term: 1 leader_uuid: "f817cd56237247ec8040a00019827479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f817cd56237247ec8040a00019827479" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 39953 } } }
I20260808 23:40:15.775154 25268 sys_catalog.cc:458] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:15.775924 25273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:15.780432 25273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:15.789758 25273 catalog_manager.cc:1347] Generated new cluster ID: 2fb3ae6f97ee4cfa9a34d867eadd0c52
I20260808 23:40:15.789965 25273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:15.802038 25273 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:15.803269 25273 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:15.810571 25273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f817cd56237247ec8040a00019827479: Generated new TSK 0
I20260808 23:40:15.811641 25273 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:40:15.870891 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:39953
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:16.197886 25288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:16.198165 25288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:16.198258 25288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:16.205657 25288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:16.205775 25288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:16.205899 25288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:16.215623 25288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:39953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:16.216861 25288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:16.218158 25288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:16.231930 25295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:16.232002 25298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:16.231973 25294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:16.233476 25288 server_base.cc:1034] running on GCE node
I20260808 23:40:16.234117 25288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:16.235100 25288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:16.236332 25288 hybrid_clock.cc:648] HybridClock initialized: now 1786232416236288 us; error 57 us; skew 500 ppm
I20260808 23:40:16.236667 25288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:16 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:16.240157 25288 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:40:16.241587 25288 webserver.cc:460] Webserver started at http://127.18.186.129:37711/ using document root <none> and password file <none>
I20260808 23:40:16.242290 25288 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:16.242462 25288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:16.242763 25288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:16.245646 25288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "0bd01b3f4f914a7e95619c0bf810204e"
format_stamp: "Formatted at 2026-08-08 23:40:16 on dist-test-slave-5k54"
server_key: "06a00506f3efb8d2472af3fe77f09351"
server_key_iv: "9e0df6a17fd967dcf2af7595346823cd"
server_key_version: "encryptionkey@0"
I20260808 23:40:16.246352 25288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "0bd01b3f4f914a7e95619c0bf810204e"
format_stamp: "Formatted at 2026-08-08 23:40:16 on dist-test-slave-5k54"
server_key: "06a00506f3efb8d2472af3fe77f09351"
server_key_iv: "9e0df6a17fd967dcf2af7595346823cd"
server_key_version: "encryptionkey@0"
I20260808 23:40:16.251272 25288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260808 23:40:16.254773 25305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:16.256100 25288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:16.256275 25288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "0bd01b3f4f914a7e95619c0bf810204e"
format_stamp: "Formatted at 2026-08-08 23:40:16 on dist-test-slave-5k54"
server_key: "06a00506f3efb8d2472af3fe77f09351"
server_key_iv: "9e0df6a17fd967dcf2af7595346823cd"
server_key_version: "encryptionkey@0"
I20260808 23:40:16.256485 25288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:16.281293 25288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:16.285262 25288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:16.285555 25288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:16.286471 25288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:16.287900 25288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:16.288004 25288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:16.288148 25288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:16.288219 25288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:16.320724 25288 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:35171
I20260808 23:40:16.320771 25427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:35171 every 8 connection(s)
I20260808 23:40:16.322011 25288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:16.324177 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 25288
I20260808 23:40:16.324402 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:40:16.325297 19178 external_mini_cluster.cc:1428] Setting key 2c8a2f2cd9c592f86d00d9d45ddab97b
Aug 08 23:40:16 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:16.346194 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:16.326097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:33485 (local address 127.18.186.190:39953)
0808 23:40:16.331894 (+ 5797us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:16.331899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:16.331924 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:40:16.331978 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:16.331982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:16.332042 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:16.332168 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:16.333792 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:16.334543 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:16.336261 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:16.336672 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:16.336811 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:16.340565 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:16.340571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:16.340574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:16.340606 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:16.343099 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:16.343896 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:16.343902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:16.343905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:16.344021 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:16.344464 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:16.344473 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:16.344475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:16.344727 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:16.344900 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:16.345601 (+ 701us) server_negotiation.cc:299] Negotiation successful
0808 23:40:16.345818 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5674,"thread_start_us":5484,"threads_started":1}
I20260808 23:40:16.347136 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:16.325500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:39953 (local address 127.18.186.129:33485)
0808 23:40:16.326413 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:16.326476 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:16.328468 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:16.332444 (+ 3976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:16.332454 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:16.333002 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:16.333605 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:16.333624 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:16.334724 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:16.334729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:16.336105 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:16.336117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:16.336917 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:16.336925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:16.337171 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:16.337913 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:16.337936 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:16.340341 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:16.343240 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:16.343249 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:16.343266 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:16.343760 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:16.344132 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:16.344137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:16.344141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:16.344333 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:16.344880 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:16.344888 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:16.345225 (+ 337us) client_negotiation.cc:769] Sending connection context
0808 23:40:16.345489 (+ 264us) client_negotiation.cc:240] Negotiation successful
0808 23:40:16.346039 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":341,"threads_started":1}
I20260808 23:40:16.348666 25428 heartbeater.cc:347] Connected to a master server at 127.18.186.190:39953
I20260808 23:40:16.348974 25428 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:16.349700 25428 heartbeater.cc:511] Master 127.18.186.190:39953 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
I20260808 23:40:16.351852 25223 ts_manager.cc:194] Registered new tserver with Master: 0bd01b3f4f914a7e95619c0bf810204e (127.18.186.129:35171)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
I20260808 23:40:16.353327 25223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:33485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:40:16.378690 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:39953
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:16.715842 25435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:16.716106 25435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:16.716287 25435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:16.723412 25435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:16.723551 25435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:16.723757 25435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:16.733408 25435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:39953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:16.734630 25435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:16.735929 25435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:16.750213 25440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:16.750275 25441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:16.750322 25443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:16.751292 25435 server_base.cc:1034] running on GCE node
I20260808 23:40:16.751991 25435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:16.752878 25435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:16.754107 25435 hybrid_clock.cc:648] HybridClock initialized: now 1786232416754051 us; error 30 us; skew 500 ppm
I20260808 23:40:16.754459 25435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:16 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:16.757648 25435 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:40:16.759145 25435 webserver.cc:460] Webserver started at http://127.18.186.130:46781/ using document root <none> and password file <none>
I20260808 23:40:16.759975 25435 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:16.760109 25435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:16.760432 25435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:16.763397 25435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "4eb04be3b5ed4316b8d55f94763fd91e"
format_stamp: "Formatted at 2026-08-08 23:40:16 on dist-test-slave-5k54"
server_key: "fd57de9cec9ad6b48b526b7b7661ed13"
server_key_iv: "142ecf923edbaeea9aae3a7d3c5ae7e6"
server_key_version: "encryptionkey@0"
I20260808 23:40:16.764206 25435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "4eb04be3b5ed4316b8d55f94763fd91e"
format_stamp: "Formatted at 2026-08-08 23:40:16 on dist-test-slave-5k54"
server_key: "fd57de9cec9ad6b48b526b7b7661ed13"
server_key_iv: "142ecf923edbaeea9aae3a7d3c5ae7e6"
server_key_version: "encryptionkey@0"
I20260808 23:40:16.769155 25435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:40:16.772543 25450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:16.773805 25435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:40:16.773988 25435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "4eb04be3b5ed4316b8d55f94763fd91e"
format_stamp: "Formatted at 2026-08-08 23:40:16 on dist-test-slave-5k54"
server_key: "fd57de9cec9ad6b48b526b7b7661ed13"
server_key_iv: "142ecf923edbaeea9aae3a7d3c5ae7e6"
server_key_version: "encryptionkey@0"
I20260808 23:40:16.774185 25435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:16.785424 25435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:16.789422 25435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:16.789706 25435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:16.790601 25435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:16.791923 25435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:16.792003 25435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:16.792171 25435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:16.792223 25435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:16.826186 25435 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:43269
I20260808 23:40:16.826290 25572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:43269 every 8 connection(s)
I20260808 23:40:16.827574 25435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:16.829519 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 25435
I20260808 23:40:16.829711 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:40:16.830188 19178 external_mini_cluster.cc:1428] Setting key d77df4b6c6b0fc9ea17841515c4bc739
Aug 08 23:40:16 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:16.852118 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:16.830426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:35035 (local address 127.18.186.190:39953)
0808 23:40:16.830646 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:16.830653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:16.832036 (+ 1383us) server_negotiation.cc:407] Connection header received
0808 23:40:16.834851 (+ 2815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:16.834855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:16.834923 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:16.835010 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:16.837168 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:16.837892 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:16.840395 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:16.840834 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:16.840930 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:16.844815 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:16.844824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:16.844831 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:16.844879 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:16.848658 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:16.849413 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:16.849422 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:16.849428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:16.849586 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:16.850049 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:16.850057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:16.850062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:16.850483 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:16.850669 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:16.851341 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 23:40:16.851656 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:40:16.852875 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:16.830911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:39953 (local address 127.18.186.130:35035)
0808 23:40:16.831847 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:16.831908 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:16.834544 (+ 2636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:16.835283 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:16.835298 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:16.836116 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:16.836937 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:16.836965 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:16.838058 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:16.838066 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:16.840241 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:16.840258 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:16.841061 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:16.841071 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:16.841415 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:16.842235 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:16.842258 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:16.844541 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:16.848803 (+ 4262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:16.848810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:16.848822 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:16.849265 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:16.849719 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:16.849723 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:16.849725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:16.849926 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:16.850615 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:16.850621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:16.850948 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 23:40:16.851174 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 23:40:16.851646 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":415,"threads_started":1}
I20260808 23:40:16.854477 25573 heartbeater.cc:347] Connected to a master server at 127.18.186.190:39953
I20260808 23:40:16.854816 25573 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:16.855602 25573 heartbeater.cc:511] Master 127.18.186.190:39953 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
I20260808 23:40:16.857157 25223 ts_manager.cc:194] Registered new tserver with Master: 4eb04be3b5ed4316b8d55f94763fd91e (127.18.186.130:43269)
I20260808 23:40:16.857955 25223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:35035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:40:16.895387 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:39953
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:17.222335 25580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:17.222618 25580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:17.222713 25580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:17.229883 25580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:17.229985 25580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:17.230095 25580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:17.239694 25580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:39953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:17.240854 25580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:17.242094 25580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:17.255177 25585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:17.255214 25586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:17.255137 25588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:17.256156 25580 server_base.cc:1034] running on GCE node
I20260808 23:40:17.256716 25580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:17.257558 25580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:17.258751 25580 hybrid_clock.cc:648] HybridClock initialized: now 1786232417258686 us; error 51 us; skew 500 ppm
I20260808 23:40:17.259130 25580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:17.262332 25580 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:40:17.263849 25580 webserver.cc:460] Webserver started at http://127.18.186.131:42643/ using document root <none> and password file <none>
I20260808 23:40:17.264544 25580 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:17.264694 25580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:17.265025 25580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:17.268055 25580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "4d24b1357dc24e9ba0a8878c6403362e"
format_stamp: "Formatted at 2026-08-08 23:40:17 on dist-test-slave-5k54"
server_key: "2cf96e220694e24aa4469e4c0aa0b7a5"
server_key_iv: "f6043bf2546e57941566a46f7e650944"
server_key_version: "encryptionkey@0"
I20260808 23:40:17.268846 25580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "4d24b1357dc24e9ba0a8878c6403362e"
format_stamp: "Formatted at 2026-08-08 23:40:17 on dist-test-slave-5k54"
server_key: "2cf96e220694e24aa4469e4c0aa0b7a5"
server_key_iv: "f6043bf2546e57941566a46f7e650944"
server_key_version: "encryptionkey@0"
I20260808 23:40:17.274241 25580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:40:17.277616 25595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:17.279052 25580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:40:17.279230 25580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "4d24b1357dc24e9ba0a8878c6403362e"
format_stamp: "Formatted at 2026-08-08 23:40:17 on dist-test-slave-5k54"
server_key: "2cf96e220694e24aa4469e4c0aa0b7a5"
server_key_iv: "f6043bf2546e57941566a46f7e650944"
server_key_version: "encryptionkey@0"
I20260808 23:40:17.279502 25580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:17.302053 25580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:17.306609 25580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:17.306905 25580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:17.307876 25580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:17.309223 25580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:17.309330 25580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:17.309484 25580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:17.309558 25580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:17.343400 25580 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:39471
I20260808 23:40:17.343513 25720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:39471 every 8 connection(s)
I20260808 23:40:17.344883 25580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:17.349627 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 25580
I20260808 23:40:17.349814 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:40:17.350301 19178 external_mini_cluster.cc:1428] Setting key 06d344082cbec8608e6cb466208a9d8f
I20260808 23:40:17.356170 25428 heartbeater.cc:503] Master 127.18.186.190:39953 was elected leader, sending a full tablet report...
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:17.365095 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:17.347485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:35735 (local address 127.18.186.190:39953)
0808 23:40:17.347685 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.347691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.348922 (+ 1231us) server_negotiation.cc:407] Connection header received
0808 23:40:17.350777 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.350784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.350918 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.351009 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.352755 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.353803 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.355297 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.355760 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.355895 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:17.359366 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:17.359371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:17.359374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:17.359411 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:17.362062 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.362776 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.362783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.362791 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.362911 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.363318 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.363324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.363328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.363662 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:17.363809 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:17.364520 (+ 711us) server_negotiation.cc:299] Negotiation successful
0808 23:40:17.364731 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:40:17.366099 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:17.348034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:39953 (local address 127.18.186.131:35735)
0808 23:40:17.348755 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:17.348797 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:17.350507 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:17.351270 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:17.351279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:17.351937 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:17.352591 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:17.352610 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.353964 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.353968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:17.355155 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:17.355166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.356015 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.356023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:17.356276 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:17.356992 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:17.357013 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:17.359174 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:17.362199 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:17.362208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:17.362223 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:17.362650 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:17.363021 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:17.363026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:17.363030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:17.363206 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:17.363796 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:17.363804 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:17.364132 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 23:40:17.364418 (+ 286us) client_negotiation.cc:240] Negotiation successful
0808 23:40:17.364939 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":291,"threads_started":1}
I20260808 23:40:17.367336 25721 heartbeater.cc:347] Connected to a master server at 127.18.186.190:39953
I20260808 23:40:17.367656 25721 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:17.368335 25721 heartbeater.cc:511] Master 127.18.186.190:39953 requested a full tablet report, sending...
I20260808 23:40:17.369627 25223 ts_manager.cc:194] Registered new tserver with Master: 4d24b1357dc24e9ba0a8878c6403362e (127.18.186.131:39471)
I20260808 23:40:17.370501 25223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:35735
I20260808 23:40:17.377398 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/test-admin.keytab.
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:17.409559 19178 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 23:40:17.414888 25728 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:17.412464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.18.186.129:35171)
0808 23:40:17.413088 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.413095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.413121 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:40:17.413181 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.413188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.413403 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.413528 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.414431 (+ 903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59654: BlockingRecv error: recv got EOF from 127.0.0.1:59654 (error 108)
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":331,"threads_started":1}
W20260808 23:40:17.414868 19178 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5cc' of type FILE
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:17.416514 19178 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:17.417868 19178 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:40:17.433082 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:17.419599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59658 (local address 127.18.186.129:35171)
0808 23:40:17.419782 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.419787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.419926 (+ 139us) server_negotiation.cc:407] Connection header received
0808 23:40:17.420045 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.420049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.420103 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.420181 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.421177 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.422099 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.423317 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.423973 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.424073 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:17.427678 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:17.427689 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:17.427704 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:17.427742 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:17.430277 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.430944 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.430959 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.430961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.431083 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.431540 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.431543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.431545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.431889 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:17.432190 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:17.432522 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 23:40:17.432785 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 23:40:17.438596 25728 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:17.437080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.18.186.129:35171)
0808 23:40:17.437232 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.437236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.437446 (+ 210us) server_negotiation.cc:407] Connection header received
0808 23:40:17.437562 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.437565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.437616 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.437689 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.438322 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59664: BlockingRecv error: recv got EOF from 127.0.0.1:59664 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 23:40:17.438848 19178 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5cc' of type FILE
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:17.440995 19178 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:17.442463 19178 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:40:17.455367 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:17.443685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.18.186.129:35171)
0808 23:40:17.443842 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.443846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.444190 (+ 344us) server_negotiation.cc:407] Connection header received
0808 23:40:17.444301 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.444305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.444357 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.444437 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.445300 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.446001 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.447116 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.447571 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.447670 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:17.450833 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:17.450838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:17.450841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:17.450869 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:17.453403 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.453950 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.453955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.453958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.454093 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.454411 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.454414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.454416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.454651 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:17.454845 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:17.455013 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 23:40:17.455158 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 23:40:17.459462 25728 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:17.457781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.18.186.129:35171)
0808 23:40:17.457996 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.458004 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.458230 (+ 226us) server_negotiation.cc:407] Connection header received
0808 23:40:17.458405 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.458410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.458486 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.458582 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.459211 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59680: BlockingRecv error: recv got EOF from 127.0.0.1:59680 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 23:40:17.459980 19178 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786232366675349-19178-0/krb5kdc/krb5cc)
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:17.461508 19178 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:17.462788 19178 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:40:17.476493 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:17.464121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.18.186.129:35171)
0808 23:40:17.464310 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.464317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.464530 (+ 213us) server_negotiation.cc:407] Connection header received
0808 23:40:17.464694 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.464699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.464756 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.464862 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.465811 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.466511 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.467774 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.468201 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.468335 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:17.471835 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:17.471839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:17.471843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:17.471871 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:17.474175 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.474750 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.474754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.474756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.474914 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.475224 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.475226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.475228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.475546 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:17.475779 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:17.475970 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 23:40:17.476128 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:40:17.478268 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 25288
I20260808 23:40:17.490796 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:17.478994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.18.186.129:35171)
0808 23:40:17.479131 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.479135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.479551 (+ 416us) server_negotiation.cc:407] Connection header received
0808 23:40:17.479694 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.479698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.479752 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.479823 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.480877 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.481541 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.482967 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.483498 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.483589 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:17.485862 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:17.485866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:17.485869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:17.485896 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:17.488331 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.489244 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.489248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.489250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.489366 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.489715 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.489720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.489722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.489952 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:17.490136 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:17.490398 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 23:40:17.490552 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 23:40:17.491322 25423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:17.575137 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 25288
I20260808 23:40:17.593580 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 25435
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:40:17.608557 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:17.594487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54660 (local address 127.18.186.130:43269)
0808 23:40:17.595084 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.595091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.595118 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:40:17.595182 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.595189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.595403 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.595594 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.596492 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.597521 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.598666 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.599206 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.599305 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:17.603032 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:17.603044 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:17.603058 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:17.603090 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:17.606081 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.606674 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.606681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.606683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.606798 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.607119 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.607122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.607124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.607574 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:17.607814 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:17.608085 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 23:40:17.608274 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":295,"threads_started":1}
I20260808 23:40:17.609817 25568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:17.694761 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 25435
I20260808 23:40:17.712791 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 25580
Aug 08 23:40:17 dist-test-slave-5k54 krb5kdc[25175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:40:17.728250 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:17.713713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.18.186.131:39471)
0808 23:40:17.714311 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:17.714318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:17.714347 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:40:17.714407 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:17.714415 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:17.714619 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:17.714807 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:17.715987 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.717011 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.718282 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:17.718851 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:17.719002 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:17.723023 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:17.723035 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:17.723049 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:17.723084 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:17.725778 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.726387 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.726393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.726396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.726516 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:17.726854 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:17.726857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:17.726859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:17.727204 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:17.727502 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:17.727720 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 23:40:17.727931 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":289,"threads_started":1}
I20260808 23:40:17.729539 25716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:17.814107 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 25580
I20260808 23:40:17.831691 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 25189
I20260808 23:40:17.832890 25256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:17.936936 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 25189
2026-08-08T23:40:17Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4017 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:40:18 dist-test-slave-5k54 krb5kdc[25748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:40:18 dist-test-slave-5k54 krb5kdc[25748](info): set up 2 sockets
Aug 08 23:40:18 dist-test-slave-5k54 krb5kdc[25748](info): commencing operation
krb5kdc: starting...
W20260808 23:40:19.079380 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:40:19 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.190.keytab.
Entry for principal oryx/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.190.keytab.
I20260808 23:40:19.311368 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42189
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.190.keytab
--principal=oryx/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:19.638985 25762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:19.639262 25762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:19.639474 25762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:19.646688 25762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:19.646793 25762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:19.646893 25762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:19.646974 25762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:19.647034 25762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:19.657714 25762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.190.keytab
--principal=oryx/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:19.659077 25762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:19.660410 25762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:19.671792 25770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:19.671847 25771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:19.671869 25773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:19.672698 25762 server_base.cc:1034] running on GCE node
I20260808 23:40:19.673395 25762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:19.674701 25762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:19.676059 25762 hybrid_clock.cc:648] HybridClock initialized: now 1786232419675989 us; error 51 us; skew 500 ppm
I20260808 23:40:19.676451 25762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:19 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:19.679682 25762 init.cc:377] Logged in from keytab as oryx/127.18.186.190@KRBTEST.COM (short username oryx)
I20260808 23:40:19.681140 25762 webserver.cc:460] Webserver started at http://127.18.186.190:41697/ using document root <none> and password file <none>
I20260808 23:40:19.681815 25762 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:19.681918 25762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:19.682160 25762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:19.685181 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "b98e6c34670f4565a3da3894ac0502c9"
format_stamp: "Formatted at 2026-08-08 23:40:19 on dist-test-slave-5k54"
server_key: "b4b315ef26a7bca1c29259c44c229e19"
server_key_iv: "88b13eca738c65ee04e1bddb8a67efb6"
server_key_version: "encryptionkey@0"
I20260808 23:40:19.685930 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "b98e6c34670f4565a3da3894ac0502c9"
format_stamp: "Formatted at 2026-08-08 23:40:19 on dist-test-slave-5k54"
server_key: "b4b315ef26a7bca1c29259c44c229e19"
server_key_iv: "88b13eca738c65ee04e1bddb8a67efb6"
server_key_version: "encryptionkey@0"
I20260808 23:40:19.691004 25762 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:40:19.694422 25780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:19.695765 25762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:19.695943 25762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "b98e6c34670f4565a3da3894ac0502c9"
format_stamp: "Formatted at 2026-08-08 23:40:19 on dist-test-slave-5k54"
server_key: "b4b315ef26a7bca1c29259c44c229e19"
server_key_iv: "88b13eca738c65ee04e1bddb8a67efb6"
server_key_version: "encryptionkey@0"
I20260808 23:40:19.696141 25762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:19.708076 25762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:19.711912 25762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:19.712195 25762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:19.729426 25762 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42189
I20260808 23:40:19.729455 25835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42189 every 8 connection(s)
I20260808 23:40:19.730792 25762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:19.732424 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 25762
I20260808 23:40:19.732635 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:19.733328 19178 external_mini_cluster.cc:1428] Setting key 9e993fc50c8d968be8b873ee6608b433
I20260808 23:40:19.734916 25836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:19.748693 25836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: Bootstrap starting.
Aug 08 23:40:19 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:19.752641 25836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:19.754005 25836 log.cc:824] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:19.757112 25836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: No bootstrap required, opened a new log
I20260808 23:40:19.758174 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:19.735311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60268 (local address 127.18.186.190:42189)
0808 23:40:19.736217 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:19.736232 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:19.736278 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:40:19.738222 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:19.738250 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:19.738749 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:19.739205 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:19.741026 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:19.743072 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:19.744693 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:19.745773 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:19.745967 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:19.749959 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:19.749970 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:19.749988 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:19.750047 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:19.754115 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:19.754803 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:19.754810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:19.754815 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:19.755005 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:19.755401 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:19.755408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:19.755411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:19.755980 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:19.756334 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:19.756764 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 23:40:19.757199 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":370,"threads_started":1}
I20260808 23:40:19.765195 25836 raft_consensus.cc:348] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } }
I20260808 23:40:19.765594 25836 raft_consensus.cc:374] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:19.765710 25836 raft_consensus.cc:729] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b98e6c34670f4565a3da3894ac0502c9, State: Initialized, Role: FOLLOWER
I20260808 23:40:19.766294 25836 consensus_queue.cc:260] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } }
I20260808 23:40:19.766525 25836 raft_consensus.cc:388] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:19.766645 25836 raft_consensus.cc:482] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:19.766799 25836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:19.769050 25836 raft_consensus.cc:504] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } }
I20260808 23:40:19.769546 25836 leader_election.cc:302] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b98e6c34670f4565a3da3894ac0502c9; no voters:
I20260808 23:40:19.770112 25836 leader_election.cc:288] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:19.770272 25841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:19.771081 25841 raft_consensus.cc:686] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 1 LEADER]: Becoming Leader. State: Replica: b98e6c34670f4565a3da3894ac0502c9, State: Running, Role: LEADER
I20260808 23:40:19.771715 25841 consensus_queue.cc:237] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } }
I20260808 23:40:19.772374 25836 sys_catalog.cc:564] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:19.774712 25844 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b98e6c34670f4565a3da3894ac0502c9. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } }
I20260808 23:40:19.775187 25844 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:19.775591 25842 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } }
I20260808 23:40:19.775954 25842 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:19.782827 25854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:19.788136 25854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:19.796489 25854 catalog_manager.cc:1347] Generated new cluster ID: 79856681b2aa48eda54637c35a39f9e0
I20260808 23:40:19.796593 25854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:19.820500 25854 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:19.821877 25854 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:19.829485 25854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: Generated new TSK 0
I20260808 23:40:19.830310 25854 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab.
Entry for principal oryx/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab.
I20260808 23:40:19.890697 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42189
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab
--principal=oryx/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:20.220182 25867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:20.220503 25867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:20.220670 25867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:20.228087 25867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:20.228241 25867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:20.228415 25867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:20.238281 25867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab
--principal=oryx/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:20.239683 25867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:20.241070 25867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:20.254441 25876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:20.254480 25874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:20.254504 25873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:20.255157 25867 server_base.cc:1034] running on GCE node
I20260808 23:40:20.255789 25867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:20.256742 25867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:20.257964 25867 hybrid_clock.cc:648] HybridClock initialized: now 1786232420257909 us; error 59 us; skew 500 ppm
I20260808 23:40:20.258242 25867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:20 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232420, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:20.261605 25867 init.cc:377] Logged in from keytab as oryx/127.18.186.129@KRBTEST.COM (short username oryx)
I20260808 23:40:20.263131 25867 webserver.cc:460] Webserver started at http://127.18.186.129:35627/ using document root <none> and password file <none>
I20260808 23:40:20.263895 25867 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:20.264032 25867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:20.264355 25867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:20.267525 25867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "b2e55cc3d8604e7ea8b29d95071a837a"
format_stamp: "Formatted at 2026-08-08 23:40:20 on dist-test-slave-5k54"
server_key: "89f6acc6edfa960477b29a3e5d259a29"
server_key_iv: "a0e20b32859c0d2d8793f003c46b4a6b"
server_key_version: "encryptionkey@0"
I20260808 23:40:20.268297 25867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "b2e55cc3d8604e7ea8b29d95071a837a"
format_stamp: "Formatted at 2026-08-08 23:40:20 on dist-test-slave-5k54"
server_key: "89f6acc6edfa960477b29a3e5d259a29"
server_key_iv: "a0e20b32859c0d2d8793f003c46b4a6b"
server_key_version: "encryptionkey@0"
I20260808 23:40:20.273588 25867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:40:20.277181 25883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:20.278506 25867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:40:20.278676 25867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "b2e55cc3d8604e7ea8b29d95071a837a"
format_stamp: "Formatted at 2026-08-08 23:40:20 on dist-test-slave-5k54"
server_key: "89f6acc6edfa960477b29a3e5d259a29"
server_key_iv: "a0e20b32859c0d2d8793f003c46b4a6b"
server_key_version: "encryptionkey@0"
I20260808 23:40:20.278872 25867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:20.301546 25867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:20.305500 25867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:20.305789 25867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:20.306738 25867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:20.308109 25867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:20.308218 25867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:20.308391 25867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:20.308467 25867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:20.341902 25867 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:45533
I20260808 23:40:20.341950 26003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:45533 every 8 connection(s)
I20260808 23:40:20.343226 25867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:20.345566 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 25867
I20260808 23:40:20.345819 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:40:20.346621 19178 external_mini_cluster.cc:1428] Setting key a3dc86ecc7d0bc2e5d98b014770fb003
Aug 08 23:40:20 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232420, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:20.363880 26006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:20.345934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:50849 (local address 127.18.186.190:42189)
0808 23:40:20.346435 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:20.346441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:20.347522 (+ 1081us) server_negotiation.cc:407] Connection header received
0808 23:40:20.349319 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:20.349327 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:20.349412 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:20.349530 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:20.350962 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:20.352085 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:20.353582 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:20.354325 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:20.354478 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:20.357841 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:20.357846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:20.357849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:20.357880 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:20.360541 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:20.361473 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:20.361477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:20.361480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:20.361602 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:20.362083 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:20.362086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:20.362088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:20.362346 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:20.362458 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:20.363351 (+ 893us) server_negotiation.cc:299] Negotiation successful
0808 23:40:20.363617 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":238,"threads_started":1}
I20260808 23:40:20.365397 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:20.346471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.129:50849)
0808 23:40:20.347338 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:20.347390 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:20.349055 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:20.349724 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:20.349732 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:20.350253 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:20.350796 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:20.350812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:20.352218 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:20.352221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:20.353433 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:20.353445 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:20.354539 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:20.354544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:20.354734 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:20.355458 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:20.355493 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:20.357630 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:20.360682 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:20.360694 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:20.360712 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:20.361345 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:20.361712 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:20.361719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:20.361726 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:20.361966 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:20.362480 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:20.362491 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:20.362935 (+ 444us) client_negotiation.cc:769] Sending connection context
0808 23:40:20.363247 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 23:40:20.363931 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":374,"threads_started":1}
I20260808 23:40:20.367038 26004 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42189
I20260808 23:40:20.367355 26004 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:20.368230 26004 heartbeater.cc:511] Master 127.18.186.190:42189 requested a full tablet report, sending...
I20260808 23:40:20.370358 25797 ts_manager.cc:194] Registered new tserver with Master: b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533)
WARNING: no policy specified for oryx/127.18.186.130@KRBTEST.COM; defaulting to no policy
I20260808 23:40:20.371856 25797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:50849
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab.
Entry for principal oryx/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab.
I20260808 23:40:20.402355 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42189
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab
--principal=oryx/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:20.728358 26010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:20.728672 26010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:20.728852 26010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:20.736032 26010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:20.736167 26010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:20.736344 26010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:20.745934 26010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab
--principal=oryx/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:20.747169 26010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:20.748549 26010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:20.762491 26019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:20.762542 26016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:20.762593 26017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:20.762858 26010 server_base.cc:1034] running on GCE node
I20260808 23:40:20.763810 26010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:20.764752 26010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:20.765995 26010 hybrid_clock.cc:648] HybridClock initialized: now 1786232420765938 us; error 44 us; skew 500 ppm
I20260808 23:40:20.766300 26010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:20 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232420, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:20.772410 26010 init.cc:377] Logged in from keytab as oryx/127.18.186.130@KRBTEST.COM (short username oryx)
I20260808 23:40:20.773983 26010 webserver.cc:460] Webserver started at http://127.18.186.130:40331/ using document root <none> and password file <none>
I20260808 23:40:20.774623 26010 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:20.774722 26010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:20.774983 26010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:20.778064 26010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "69dc5d50059845138c68d58c469763e5"
format_stamp: "Formatted at 2026-08-08 23:40:20 on dist-test-slave-5k54"
server_key: "155052dad1e8e0d8dc05eaa4b232b9d8"
server_key_iv: "228974170093fdb25f67451a5dce759a"
server_key_version: "encryptionkey@0"
I20260808 23:40:20.778770 26010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "69dc5d50059845138c68d58c469763e5"
format_stamp: "Formatted at 2026-08-08 23:40:20 on dist-test-slave-5k54"
server_key: "155052dad1e8e0d8dc05eaa4b232b9d8"
server_key_iv: "228974170093fdb25f67451a5dce759a"
server_key_version: "encryptionkey@0"
I20260808 23:40:20.784209 26010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 23:40:20.787570 26027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:20.788964 26010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:40:20.789151 26010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "69dc5d50059845138c68d58c469763e5"
format_stamp: "Formatted at 2026-08-08 23:40:20 on dist-test-slave-5k54"
server_key: "155052dad1e8e0d8dc05eaa4b232b9d8"
server_key_iv: "228974170093fdb25f67451a5dce759a"
server_key_version: "encryptionkey@0"
I20260808 23:40:20.789351 26010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:20.809556 26010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:20.813314 26010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:20.813596 26010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:20.814520 26010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:20.815896 26010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:20.816020 26010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:20.816169 26010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:20.816241 26010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:20.849195 26010 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:39299
I20260808 23:40:20.849300 26149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:39299 every 8 connection(s)
I20260808 23:40:20.850490 26010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:20.854784 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 26010
I20260808 23:40:20.854987 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:40:20.855563 19178 external_mini_cluster.cc:1428] Setting key 3f7a78f0fbc2caf2f62fc08e981893f2
Aug 08 23:40:20 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232420, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:20.874292 26006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:20.852998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:52239 (local address 127.18.186.190:42189)
0808 23:40:20.853214 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:20.853220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:20.854369 (+ 1149us) server_negotiation.cc:407] Connection header received
0808 23:40:20.856699 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:20.856704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:20.856778 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:20.856882 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:20.858728 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:20.859625 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:20.862030 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:20.862479 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:20.862569 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:20.866733 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:20.866737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:20.866740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:20.866772 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:20.869776 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:20.870812 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:20.870820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:20.870844 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:20.871028 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:20.871776 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:20.871783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:20.871788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:20.872175 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:20.872357 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:20.873617 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0808 23:40:20.873848 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":71}
I20260808 23:40:20.876214 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:20.853504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.130:52239)
0808 23:40:20.854197 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:20.854235 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:20.856438 (+ 2203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:20.857082 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:20.857090 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:20.857684 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:20.858493 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:20.858523 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:20.859780 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:20.859789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:20.861879 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:20.861897 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:20.862710 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:20.862721 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:20.863041 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:20.863795 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:20.863834 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:20.866509 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:20.869955 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:20.869969 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:20.869993 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:20.870664 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:20.871171 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:20.871178 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:20.871185 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:20.871645 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:20.872326 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:20.872337 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:20.873056 (+ 719us) client_negotiation.cc:769] Sending connection context
0808 23:40:20.873495 (+ 439us) client_negotiation.cc:240] Negotiation successful
0808 23:40:20.874312 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":263,"threads_started":1}
I20260808 23:40:20.878216 26150 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42189
I20260808 23:40:20.878567 26150 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:20.879343 26150 heartbeater.cc:511] Master 127.18.186.190:42189 requested a full tablet report, sending...
I20260808 23:40:20.880956 25797 ts_manager.cc:194] Registered new tserver with Master: 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299)
I20260808 23:40:20.881814 25797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:52239
WARNING: no policy specified for oryx/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab.
Entry for principal oryx/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab.
I20260808 23:40:20.914762 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42189
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab
--principal=oryx/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:21.238056 26155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:21.238390 26155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:21.238577 26155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:21.245952 26155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:21.246115 26155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:21.246307 26155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:21.256119 26155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab
--principal=oryx/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:21.257434 26155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:21.258806 26155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:21.271891 26162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:21.271966 26161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:21.271940 26164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:21.273100 26155 server_base.cc:1034] running on GCE node
I20260808 23:40:21.273808 26155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:21.274768 26155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:21.275990 26155 hybrid_clock.cc:648] HybridClock initialized: now 1786232421275925 us; error 60 us; skew 500 ppm
I20260808 23:40:21.276300 26155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:21 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232421, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:21.279567 26155 init.cc:377] Logged in from keytab as oryx/127.18.186.131@KRBTEST.COM (short username oryx)
I20260808 23:40:21.281117 26155 webserver.cc:460] Webserver started at http://127.18.186.131:32973/ using document root <none> and password file <none>
I20260808 23:40:21.281813 26155 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:21.281963 26155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:21.282327 26155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:21.285501 26155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "dfa6b0c648d64bbe8675e8f4cc891256"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "03ba9ed4d1350bc79f63fe079df6ac8f"
server_key_iv: "f980aded583f43d478d3e625a82134de"
server_key_version: "encryptionkey@0"
I20260808 23:40:21.286293 26155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "dfa6b0c648d64bbe8675e8f4cc891256"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "03ba9ed4d1350bc79f63fe079df6ac8f"
server_key_iv: "f980aded583f43d478d3e625a82134de"
server_key_version: "encryptionkey@0"
I20260808 23:40:21.291326 26155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 23:40:21.294967 26172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:21.296406 26155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:40:21.296629 26155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "dfa6b0c648d64bbe8675e8f4cc891256"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "03ba9ed4d1350bc79f63fe079df6ac8f"
server_key_iv: "f980aded583f43d478d3e625a82134de"
server_key_version: "encryptionkey@0"
I20260808 23:40:21.296875 26155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:21.324199 26155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:21.328691 26155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:21.329005 26155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:21.330004 26155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:21.331398 26155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:21.331537 26155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:21.331691 26155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:21.331761 26155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:21.366039 26155 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:37883
I20260808 23:40:21.366086 26294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:37883 every 8 connection(s)
I20260808 23:40:21.367316 26155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:21.367928 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 26155
I20260808 23:40:21.368149 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:40:21.368831 19178 external_mini_cluster.cc:1428] Setting key 2990b4fefb1f21edb549d42db7dc86a5
I20260808 23:40:21.375041 26004 heartbeater.cc:503] Master 127.18.186.190:42189 was elected leader, sending a full tablet report...
Aug 08 23:40:21 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232421, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:21.388916 26006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.370402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:54915 (local address 127.18.186.190:42189)
0808 23:40:21.370621 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:21.370627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:21.371889 (+ 1262us) server_negotiation.cc:407] Connection header received
0808 23:40:21.374106 (+ 2217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:21.374110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:21.374203 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:21.374284 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:21.375990 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.377143 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.378649 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.379074 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.379199 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.382820 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:21.382827 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:21.382832 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:21.382876 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:21.386238 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.386891 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.386895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.386897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.387018 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.387398 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.387401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.387403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.387733 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:21.387856 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:21.388492 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 23:40:21.388654 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 23:40:21.389842 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.370872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.131:54915)
0808 23:40:21.371714 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:21.371767 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:21.373862 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:21.374502 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:21.374511 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:21.375017 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:21.375815 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.375846 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.377289 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.377294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.378505 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.378521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.379301 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.379308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.379595 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.380209 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:21.380230 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:21.382609 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:21.386355 (+ 3746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.386361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.386373 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.386783 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.387105 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.387108 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.387111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.387286 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.387837 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:21.387843 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:21.388145 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 23:40:21.388363 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 23:40:21.388728 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":345,"threads_started":1}
I20260808 23:40:21.391196 26295 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42189
I20260808 23:40:21.391554 26295 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:21.392297 26295 heartbeater.cc:511] Master 127.18.186.190:42189 requested a full tablet report, sending...
I20260808 23:40:21.393751 25797 ts_manager.cc:194] Registered new tserver with Master: dfa6b0c648d64bbe8675e8f4cc891256 (127.18.186.131:37883)
I20260808 23:40:21.394614 25797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:54915
I20260808 23:40:21.397782 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.186.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.189.keytab.
Entry for principal oryx/127.18.186.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.189.keytab.
I20260808 23:40:21.451529 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.189:39769
--webserver_interface=127.18.186.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.189.keytab
--principal=oryx/127.18.186.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42189,127.18.186.189:39769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:21.784081 26301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:21.784351 26301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:21.784536 26301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:21.791714 26301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:21.791821 26301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:21.791960 26301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:21.792058 26301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:21.792107 26301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:21.802469 26301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42189,127.18.186.189:39769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.189.keytab
--principal=oryx/127.18.186.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.189:39769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.186.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
W20260808 23:40:21.802781 26301 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.186.190:42189,127.18.186.189:39769'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 23:40:21.803861 26301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:21.805160 26301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:21.816694 26307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:21.816766 26309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:21.816741 26306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:21.817853 26301 server_base.cc:1034] running on GCE node
I20260808 23:40:21.818627 26301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:21.820055 26301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:21.821273 26301 hybrid_clock.cc:648] HybridClock initialized: now 1786232421821193 us; error 52 us; skew 500 ppm
I20260808 23:40:21.821604 26301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:21 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232421, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:21.824791 26301 init.cc:377] Logged in from keytab as oryx/127.18.186.189@KRBTEST.COM (short username oryx)
I20260808 23:40:21.826220 26301 webserver.cc:460] Webserver started at http://127.18.186.189:43215/ using document root <none> and password file <none>
I20260808 23:40:21.826929 26301 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:21.827059 26301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:21.827363 26301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:21.830308 26301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data/instance:
uuid: "fb4bc7131cf44e9a87eea91b293def91"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "e4e8494752ed54345ea79973d3474c03"
server_key_iv: "71bfd98d5bc9a6025b33f9743bc029d3"
server_key_version: "encryptionkey@0"
I20260808 23:40:21.831100 26301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal/instance:
uuid: "fb4bc7131cf44e9a87eea91b293def91"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "e4e8494752ed54345ea79973d3474c03"
server_key_iv: "71bfd98d5bc9a6025b33f9743bc029d3"
server_key_version: "encryptionkey@0"
I20260808 23:40:21.836591 26301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:40:21.840185 26316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:21.841509 26301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 23:40:21.841696 26301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
uuid: "fb4bc7131cf44e9a87eea91b293def91"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "e4e8494752ed54345ea79973d3474c03"
server_key_iv: "71bfd98d5bc9a6025b33f9743bc029d3"
server_key_version: "encryptionkey@0"
I20260808 23:40:21.841918 26301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:21.861505 26301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:21.865530 26301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:21.865787 26301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:21.884938 26150 heartbeater.cc:503] Master 127.18.186.190:42189 was elected leader, sending a full tablet report...
I20260808 23:40:21.885569 26301 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.189:39769
I20260808 23:40:21.885592 26372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.189:39769 every 8 connection(s)
I20260808 23:40:21.887027 26301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
I20260808 23:40:21.890726 26373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:21.894018 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 26301
I20260808 23:40:21.898579 26373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:40:21 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.189@KRBTEST.COM
Aug 08 23:40:21 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232421, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.189@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:21.917066 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.902298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.18.186.190:42189)
0808 23:40:21.902791 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:21.902799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:21.903554 (+ 755us) server_negotiation.cc:407] Connection header received
0808 23:40:21.903717 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:21.903721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:21.903782 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:21.903871 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:21.905154 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.905899 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.907463 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.907861 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.908001 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.911366 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:21.911371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:21.911374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:21.911412 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:21.913973 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.914689 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.914695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.914698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.914829 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.915192 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.915197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.915200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.915591 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:21.915742 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:21.916285 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 23:40:21.916538 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":263,"threads_started":1}
I20260808 23:40:21.917586 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.902259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:60282)
0808 23:40:21.903346 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:21.903452 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:21.903606 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:21.903951 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:21.903959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:21.904742 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:21.905001 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.905010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.906046 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.906052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.907286 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.907297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.908094 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.908101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.908254 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.908848 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:21.908868 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:21.911181 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:21.914105 (+ 2924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.914116 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.914126 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.914555 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.914942 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.914947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.914950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.915087 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.915735 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:21.915743 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:21.915930 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 23:40:21.916192 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 23:40:21.916741 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"mutex_wait_us":1,"spinlock_wait_cycles":14208,"thread_start_us":678,"threads_started":1}
I20260808 23:40:21.917606 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.896300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.18.186.189:39769)
0808 23:40:21.897065 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:21.897079 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:21.897127 (+ 48us) server_negotiation.cc:407] Connection header received
0808 23:40:21.899358 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:21.899404 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:21.899954 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:21.900398 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:21.902046 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.904107 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.905377 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.906011 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.906238 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.910517 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:21.910524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:21.910535 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:21.910563 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:21.913593 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.914241 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.914257 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.914259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.914426 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.914832 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.914838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.914841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.915291 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:21.915648 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:21.916102 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 23:40:21.916524 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":541,"thread_start_us":326,"threads_started":1}
I20260808 23:40:21.920264 26373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:40:21 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232421, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.189@KRBTEST.COM for oryx/127.18.186.189@KRBTEST.COM
I20260808 23:40:21.932217 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.921045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.0.0.1:49294)
0808 23:40:21.921229 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:21.921248 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:21.921344 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:21.921633 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:21.921637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:21.921999 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:21.922124 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.922131 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.923059 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.923064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.923976 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.923987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.924622 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.924627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.924868 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.925400 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:21.925417 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:21.927268 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:21.929888 (+ 2620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.929892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.929896 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.930251 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.930592 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.930598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.930602 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.930727 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.931216 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:21.931220 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:21.931374 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 23:40:21.931577 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 23:40:21.931859 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 23:40:21.932300 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.921149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.18.186.189:39769)
0808 23:40:21.921289 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:21.921294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:21.921310 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:40:21.921466 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:21.921471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:21.921526 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:21.921618 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:21.922249 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.922922 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.924105 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.924486 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.924614 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.927468 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:21.927473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:21.927477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:21.927503 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:21.929749 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.930364 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.930372 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.930375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.930488 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.930825 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.930832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.930835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.931103 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:21.931254 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:21.931651 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 23:40:21.932008 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":199,"server-negotiator.queue_time_us":48}
I20260808 23:40:21.937186 26373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Bootstrap starting.
I20260808 23:40:21.940284 26373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:21.941288 26373 log.cc:824] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:21.944003 26373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: No bootstrap required, opened a new log
I20260808 23:40:21.952028 26373 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } }
I20260808 23:40:21.952349 26373 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:21.952474 26373 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4bc7131cf44e9a87eea91b293def91, State: Initialized, Role: FOLLOWER
I20260808 23:40:21.953155 26373 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } }
I20260808 23:40:21.954015 26382 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } } }
I20260808 23:40:21.954303 26382 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:40:21.955003 26373 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 23:40:21.964207 26396 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:40:21.964344 26396 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:40:21.967010 26301 master_runner.cc:428] Detected that this master fb4bc7131cf44e9a87eea91b293def91 is joining an existing cluster
I20260808 23:40:21.967110 26301 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.186.189:39769
I20260808 23:40:21.990772 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.971399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60298 (local address 127.18.186.190:42189)
0808 23:40:21.971928 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:21.971939 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:21.972601 (+ 662us) server_negotiation.cc:407] Connection header received
0808 23:40:21.972940 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:21.972947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:21.973060 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:21.973141 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:21.975130 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.976633 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.979371 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.980224 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.980330 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.983168 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:21.983175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:21.983180 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:21.983221 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:21.987076 (+ 3855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.987960 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.987965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.987969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.988126 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.988614 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.988619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.988623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.989012 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:21.989141 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:21.990159 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0808 23:40:21.990414 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388}
I20260808 23:40:21.993073 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.971512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:60298)
0808 23:40:21.972486 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:21.972508 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:21.972751 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:21.973365 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:21.973370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:21.974342 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:21.974901 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.974919 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.976866 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.976875 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.979145 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.979166 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.980650 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.980656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.980850 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.981600 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:21.981621 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:21.982960 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:21.987222 (+ 4262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.987228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.987232 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.987796 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.988288 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.988293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.988296 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.988471 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.989236 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:21.989241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:21.989792 (+ 551us) client_negotiation.cc:769] Sending connection context
0808 23:40:21.991168 (+ 1376us) client_negotiation.cc:240] Negotiation successful
0808 23:40:21.992537 (+ 1369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"spinlock_wait_cycles":42752,"thread_start_us":480,"threads_started":1}
I20260808 23:40:21.999030 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.973062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49300 (local address 127.18.186.189:39769)
0808 23:40:21.973256 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:21.973261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:21.980379 (+ 7118us) server_negotiation.cc:407] Connection header received
0808 23:40:21.980818 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:21.980823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:21.980890 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:21.980958 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:21.982464 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.983585 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.987571 (+ 3986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.988456 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.988580 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.991177 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:21.991185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:21.991193 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:21.991232 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:21.995486 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.996472 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.996480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.996486 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.996647 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:21.997121 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:21.997128 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:21.997130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:21.997567 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:21.997706 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:21.998289 (+ 583us) server_negotiation.cc:299] Negotiation successful
0808 23:40:21.998553 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 23:40:22.000069 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:21.973562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.0.0.1:49300)
0808 23:40:21.980294 (+ 6732us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:21.980313 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:21.980689 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:21.981156 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:21.981160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:21.981896 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:21.982303 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.982321 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.983812 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.983819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.985474 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:21.985492 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:21.988911 (+ 3419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:21.988920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:21.989136 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:21.989744 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:21.989787 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:21.991008 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:21.995630 (+ 4622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.995638 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.995644 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.996347 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.996790 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:21.996797 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:21.996804 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:21.997000 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:21.997800 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:21.997805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:21.997972 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 23:40:21.999222 (+ 1250us) client_negotiation.cc:240] Negotiation successful
0808 23:40:21.999560 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6588,"spinlock_wait_cycles":7424,"thread_start_us":6820,"threads_started":1}
WARNING: no policy specified for oryx/127.18.186.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.188@KRBTEST.COM" created.
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.190@KRBTEST.COM for oryx/127.18.186.189@KRBTEST.COM
I20260808 23:40:22.019840 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.001879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.18.186.189:39769)
0808 23:40:22.002035 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.002040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.002299 (+ 259us) server_negotiation.cc:407] Connection header received
0808 23:40:22.002569 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.002575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.002695 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.002798 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.004389 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.005498 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.006838 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.007606 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.007734 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.012191 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.012200 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.012208 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.012249 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.016371 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.017285 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.017291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.017295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.017455 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.017982 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.017987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.017990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.018333 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.018454 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.019164 (+ 710us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.019394 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:40:22.020954 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.001499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.0.0.1:49308)
0808 23:40:22.002143 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.002199 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.002371 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.002880 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.002898 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.003859 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.004221 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.004237 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.005637 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.005645 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.006672 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.006690 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.007965 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.007972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.008183 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.008928 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.008954 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.011984 (+ 3030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.016527 (+ 4543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.016542 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.016550 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.017133 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.017626 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.017631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.017634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.017825 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.018585 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.018594 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.018793 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.020091 (+ 1298us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.020463 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":543,"threads_started":1}
I20260808 23:40:22.023406 25797 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.186.189:39769: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91"
member_type: NON_VOTER
last_known_addr {
host: "127.18.186.189"
port: 39769
}
attrs {
promote: true
}
}
dest_uuid: "b98e6c34670f4565a3da3894ac0502c9"
cas_config_opid_index: -1
I20260808 23:40:22.025058 25797 consensus_queue.cc:237] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } }
I20260808 23:40:22.034646 26412 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fb4bc7131cf44e9a87eea91b293def91 (127.18.186.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } }
I20260808 23:40:22.036485 26346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:22.039258 26412 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.186.189:39769
I20260808 23:40:22.040254 26301 master.cc:537] Master@127.18.186.189:39769 shutting down...
I20260808 23:40:22.040350 26411 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } }
I20260808 23:40:22.040693 26411 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:22.042402 26346 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Refusing update from remote peer b98e6c34670f4565a3da3894ac0502c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 23:40:22.044526 26411 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.188.keytab.
Entry for principal oryx/127.18.186.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.188.keytab.
I20260808 23:40:22.048065 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.188:39493
--webserver_interface=127.18.186.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.188.keytab
--principal=oryx/127.18.186.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42189,127.18.186.189:39769,127.18.186.188:39493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:22.048413 25781 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 23:40:22.050098 25781 consensus_peers.cc:487] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 -> Peer fb4bc7131cf44e9a87eea91b293def91 (127.18.186.189:39769): Couldn't send request to peer fb4bc7131cf44e9a87eea91b293def91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 23:40:22.064375 26301 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:22.064843 26301 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:22.065061 26301 tablet_replica.cc:330] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: stopping tablet replica
I20260808 23:40:22.073043 26301 master.cc:552] Master@127.18.186.189:39769 shutdown complete.
I20260808 23:40:22.073180 26301 master_runner.cc:305] Clearing existing system tablet
I20260808 23:40:22.073655 26301 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:22.073751 26301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:22.076943 26301 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.004s
I20260808 23:40:22.078642 26417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:22.078965 26301 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:22.079234 26301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 23:40:22.079324 26301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
uuid: "fb4bc7131cf44e9a87eea91b293def91"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "e4e8494752ed54345ea79973d3474c03"
server_key_iv: "71bfd98d5bc9a6025b33f9743bc029d3"
server_key_version: "encryptionkey@0"
I20260808 23:40:22.084883 26301 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:22.091630 26301 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 23:40:22.091822 26301 log.cc:1196] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 23:40:22.092293 26301 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Deleting consensus metadata
I20260808 23:40:22.092581 26301 master_runner.cc:315] Copying system tablet from 127.18.186.190:42189
I20260808 23:40:22.095400 26301 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: tablet copy: Beginning tablet copy session from remote peer at address 127.18.186.190:42189
I20260808 23:40:22.112511 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.096438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.18.186.190:42189)
0808 23:40:22.096667 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.096673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.097275 (+ 602us) server_negotiation.cc:407] Connection header received
0808 23:40:22.097462 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.097468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.097573 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.097686 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.098872 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.100060 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.102200 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.102882 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.103010 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.105436 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.105442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.105447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.105489 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.109417 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.110254 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.110259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.110262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.110428 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.110861 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.110865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.110868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.111233 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.111400 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.111885 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.112116 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:40:22.112861 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.096391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:60314)
0808 23:40:22.097116 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.097145 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.097314 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.097745 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.097751 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.098377 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.098686 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.098700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.100305 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.100311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.102030 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.102046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.103071 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.103078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.103258 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.104058 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.104082 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.105200 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.109562 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.109568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.109573 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.110113 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.110549 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.110559 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.110563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.110736 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.111370 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.111376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.111585 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.111751 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.112117 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":413,"threads_started":1}
I20260808 23:40:22.114053 25819 tablet_copy_service.cc:140] P b98e6c34670f4565a3da3894ac0502c9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fb4bc7131cf44e9a87eea91b293def91 ({username='oryx', principal='oryx/127.18.186.189@KRBTEST.COM'} at 127.0.0.1:60314)
I20260808 23:40:22.114485 25819 tablet_copy_service.cc:161] P b98e6c34670f4565a3da3894ac0502c9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fb4bc7131cf44e9a87eea91b293def91 at {username='oryx', principal='oryx/127.18.186.189@KRBTEST.COM'} at 127.0.0.1:60314: session id = fb4bc7131cf44e9a87eea91b293def91-00000000000000000000000000000000
I20260808 23:40:22.119659 25819 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 23:40:22.122458 26301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 23:40:22.129585 26301 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: tablet copy: Starting download of 0 data blocks...
I20260808 23:40:22.129999 26301 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: tablet copy: Starting download of 1 WAL segments...
I20260808 23:40:22.133911 26301 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 23:40:22.139242 25819 tablet_copy_service.cc:342] P b98e6c34670f4565a3da3894ac0502c9: Request end of tablet copy session fb4bc7131cf44e9a87eea91b293def91-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.186.189@KRBTEST.COM'} at 127.0.0.1:60314
I20260808 23:40:22.139707 25819 tablet_copy_service.cc:433] P b98e6c34670f4565a3da3894ac0502c9: ending tablet copy session fb4bc7131cf44e9a87eea91b293def91-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fb4bc7131cf44e9a87eea91b293def91
I20260808 23:40:22.152691 26301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:22.155550 26426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:22.160812 26427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:22.167683 26429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:22.168603 26301 server_base.cc:1034] running on GCE node
I20260808 23:40:22.168987 26301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:22.169673 26301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:22.170964 26301 hybrid_clock.cc:648] HybridClock initialized: now 1786232422170900 us; error 34 us; skew 500 ppm
I20260808 23:40:22.171271 26301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232422, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:22.173604 26301 init.cc:377] Logged in from keytab as oryx/127.18.186.189@KRBTEST.COM (short username oryx)
I20260808 23:40:22.176330 26301 webserver.cc:460] Webserver started at http://127.18.186.189:44579/ using document root <none> and password file <none>
I20260808 23:40:22.176753 26301 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:22.176895 26301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:22.180183 26301 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 23:40:22.182264 26436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:22.182780 26301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 23:40:22.183048 26301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
uuid: "fb4bc7131cf44e9a87eea91b293def91"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "e4e8494752ed54345ea79973d3474c03"
server_key_iv: "71bfd98d5bc9a6025b33f9743bc029d3"
server_key_version: "encryptionkey@0"
I20260808 23:40:22.183359 26301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:22.198299 26301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:22.202714 26301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:22.228041 26301 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.189:39769
I20260808 23:40:22.228071 26491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.189:39769 every 8 connection(s)
I20260808 23:40:22.229297 26301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
I20260808 23:40:22.234989 26492 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:40:22.239303 26492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Bootstrap starting.
I20260808 23:40:22.258623 26492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:40:22.259177 26492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: Bootstrap complete.
I20260808 23:40:22.260293 26492 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } }
I20260808 23:40:22.260558 26492 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fb4bc7131cf44e9a87eea91b293def91, State: Initialized, Role: LEARNER
I20260808 23:40:22.260789 26492 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } }
I20260808 23:40:22.261564 26496 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } }
I20260808 23:40:22.261832 26492 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:22.261973 26496 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:40:22.270560 26508 catalog_manager.cc:1259] Loaded cluster ID: 79856681b2aa48eda54637c35a39f9e0
I20260808 23:40:22.270655 26508 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: loading cluster ID for follower catalog manager: success
I20260808 23:40:22.274175 26508 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: acquiring CA information for follower catalog manager: success
I20260808 23:40:22.276717 26508 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:40:22.397571 26295 heartbeater.cc:503] Master 127.18.186.190:42189 was elected leader, sending a full tablet report...
W20260808 23:40:22.459833 26415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:22.460134 26415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:22.460448 26415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:22.470623 26415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:22.470805 26415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:22.470956 26415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:22.471064 26415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:22.471148 26415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:22.483707 26415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42189,127.18.186.189:39769,127.18.186.188:39493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.188.keytab
--principal=oryx/127.18.186.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.188:39493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.186.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:22.485627 26415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:22.487752 26415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:22.505127 26513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:22.506016 26514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:22.506726 26517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:22.508988 26415 server_base.cc:1034] running on GCE node
I20260808 23:40:22.509851 26415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:22.511488 26415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:22.512066 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.498997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.0.0.1:49324)
0808 23:40:22.499193 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.499213 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.499347 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.504261 (+ 4914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.504268 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.504304 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:22.504506 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.504517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.507211 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.507218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.509413 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.509433 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.510945 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.510953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.511471 (+ 518us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.511534 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.511675 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.511829 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 23:40:22.512703 26415 hybrid_clock.cc:648] HybridClock initialized: now 1786232422512641 us; error 58 us; skew 500 ppm
I20260808 23:40:22.512970 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.499170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.18.186.189:39769)
0808 23:40:22.503722 (+ 4552us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.503730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.503767 (+ 37us) server_negotiation.cc:407] Connection header received
0808 23:40:22.503846 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.503854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.503968 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.504217 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:22.504679 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.507049 (+ 2370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.509597 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.510732 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.511144 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.511389 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.512417 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.512546 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4357,"thread_start_us":4082,"threads_started":1}
I20260808 23:40:22.513043 26415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:22.514725 26464 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 23:40:22.515609 26496 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b98e6c34670f4565a3da3894ac0502c9. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } }
I20260808 23:40:22.515990 26496 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: This master's current role is: LEARNER
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232422, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:22.518057 26415 init.cc:377] Logged in from keytab as oryx/127.18.186.188@KRBTEST.COM (short username oryx)
I20260808 23:40:22.520510 26415 webserver.cc:460] Webserver started at http://127.18.186.188:41621/ using document root <none> and password file <none>
I20260808 23:40:22.521427 26415 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:22.521627 26415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:22.522022 26415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:22.525677 26415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data/instance:
uuid: "efd512694f1846a0935af075037a343d"
format_stamp: "Formatted at 2026-08-08 23:40:22 on dist-test-slave-5k54"
server_key: "dc75a76327c47c516457bb96f6d4890e"
server_key_iv: "54e65c44cddd231c5be0663076d7bee3"
server_key_version: "encryptionkey@0"
I20260808 23:40:22.526556 26415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal/instance:
uuid: "efd512694f1846a0935af075037a343d"
format_stamp: "Formatted at 2026-08-08 23:40:22 on dist-test-slave-5k54"
server_key: "dc75a76327c47c516457bb96f6d4890e"
server_key_iv: "54e65c44cddd231c5be0663076d7bee3"
server_key_version: "encryptionkey@0"
I20260808 23:40:22.532781 26415 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260808 23:40:22.537074 26526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:22.538380 26415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:40:22.538604 26415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
uuid: "efd512694f1846a0935af075037a343d"
format_stamp: "Formatted at 2026-08-08 23:40:22 on dist-test-slave-5k54"
server_key: "dc75a76327c47c516457bb96f6d4890e"
server_key_iv: "54e65c44cddd231c5be0663076d7bee3"
server_key_version: "encryptionkey@0"
I20260808 23:40:22.538806 26415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:22.559634 26415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:22.565258 26415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:22.565716 26415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:22.606465 26415 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.188:39493
I20260808 23:40:22.606527 26580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.188:39493 every 8 connection(s)
I20260808 23:40:22.608474 26415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
I20260808 23:40:22.613416 26581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:22.617792 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 26415
I20260808 23:40:22.624377 26581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.188@KRBTEST.COM
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232422, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.188@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:22.651237 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.627828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.18.186.190:42189)
0808 23:40:22.628649 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.628657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.630236 (+ 1579us) server_negotiation.cc:407] Connection header received
0808 23:40:22.630487 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.630518 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.630630 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.630834 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.633345 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.634179 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.636418 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.636998 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.637096 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.642359 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.642365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.642370 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.642430 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.647277 (+ 4847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.648443 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.648446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.648449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.648616 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.649080 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.649085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.649088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.649503 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.649646 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.650445 (+ 799us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.650669 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":527,"threads_started":1}
I20260808 23:40:22.652179 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.620962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.18.186.188:39493)
0808 23:40:22.622444 (+ 1482us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.622456 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.622500 (+ 44us) server_negotiation.cc:407] Connection header received
0808 23:40:22.624872 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.624895 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.625363 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.625817 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.628133 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.631388 (+ 3255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.633982 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.635032 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.635164 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.641979 (+ 6815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.641991 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.642004 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.642035 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.647004 (+ 4969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.648008 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.648014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.648016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.648191 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.648662 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.648669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.648674 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.649415 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.649791 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.650305 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.650748 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":1180,"thread_start_us":861,"threads_started":1}
I20260808 23:40:22.652292 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.627953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:60322)
0808 23:40:22.629961 (+ 2008us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.630019 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.630284 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.631019 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.631028 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.632290 (+ 1262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.633167 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.633183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.634322 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.634326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.636266 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.636281 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.637333 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.637344 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.637653 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.638501 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.638537 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.642128 (+ 3591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.647491 (+ 5363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.647504 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.647517 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.648287 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.648731 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.648735 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.648737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.648947 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.649634 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.649648 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.649980 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.650301 (+ 321us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.650896 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1434,"thread_start_us":1032,"threads_started":1}
I20260808 23:40:22.654156 26581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232422, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.188@KRBTEST.COM for oryx/127.18.186.189@KRBTEST.COM
I20260808 23:40:22.669196 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.654944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.18.186.189:39769)
0808 23:40:22.655116 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.655120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.655512 (+ 392us) server_negotiation.cc:407] Connection header received
0808 23:40:22.655757 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.655763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.655849 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.655998 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.656681 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.657599 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.658948 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.659584 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.659681 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.663239 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.663244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.663247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.663279 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.666436 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.667270 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.667273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.667275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.667512 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.667885 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.667887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.667889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.668212 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.668311 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.668645 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.668866 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:40:22.669418 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.655154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.0.0.1:49340)
0808 23:40:22.655355 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.655369 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.655563 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.656026 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.656029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.656397 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.656560 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.656568 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.657724 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.657727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.658769 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.658780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.659729 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.659734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.659878 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.660620 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.660637 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.663012 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.666562 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.666566 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.666568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.667160 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.667606 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.667609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.667611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.667763 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.668321 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.668324 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.668461 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.668565 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.668994 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 23:40:22.670673 26581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232422, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.188@KRBTEST.COM for oryx/127.18.186.188@KRBTEST.COM
I20260808 23:40:22.685763 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.671348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:39493 (local address 127.0.0.1:49098)
0808 23:40:22.671557 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.671576 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.671707 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.671990 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.671995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.672406 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.672649 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.672659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.673701 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.673708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.674775 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.674787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.675739 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.675745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.676059 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.676680 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.676696 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.678984 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.682766 (+ 3782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.682769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.682772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.683598 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.684015 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.684018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.684021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.684203 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.684847 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.684850 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.685001 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.685109 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.685313 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 23:40:22.685770 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.671402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.18.186.188:39493)
0808 23:40:22.671558 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.671565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.671648 (+ 83us) server_negotiation.cc:407] Connection header received
0808 23:40:22.671806 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.671813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.671865 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.671960 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.672772 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.673565 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.674915 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.675553 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.675711 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.679178 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.679184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.679187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.679214 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.682629 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.683711 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.683715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.683717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.683892 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.684295 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.684298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.684300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.684735 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.684859 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.685218 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.685439 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":60}
I20260808 23:40:22.691519 26581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Bootstrap starting.
I20260808 23:40:22.695714 26581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:22.697379 26581 log.cc:824] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:22.701455 26581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: No bootstrap required, opened a new log
I20260808 23:40:22.711802 26581 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } }
I20260808 23:40:22.712219 26581 raft_consensus.cc:374] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:22.712440 26581 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd512694f1846a0935af075037a343d, State: Initialized, Role: FOLLOWER
I20260808 23:40:22.713120 26581 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } }
I20260808 23:40:22.714655 26588 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } } }
I20260808 23:40:22.715324 26588 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:40:22.715646 26581 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: configured and running, proceeding with master startup.
W20260808 23:40:22.730855 26599 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:40:22.731022 26599 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:40:22.735145 26415 master_runner.cc:428] Detected that this master efd512694f1846a0935af075037a343d is joining an existing cluster
I20260808 23:40:22.735400 26415 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.186.188:39493
I20260808 23:40:22.759943 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 25867
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.129@KRBTEST.COM
I20260808 23:40:22.789453 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.742871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:60336)
0808 23:40:22.756733 (+ 13862us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.756762 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.757090 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.757677 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.757682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.760459 (+ 2777us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.761121 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.761143 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.763303 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.763308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.771331 (+ 8023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.771352 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.774748 (+ 3396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.774754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.775040 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.776210 (+ 1170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.776233 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.779908 (+ 3675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.785337 (+ 5429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.785342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.785345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.785978 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.786480 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.786484 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.786487 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.786861 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.787715 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.787734 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.788128 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.788303 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.788704 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13654,"mutex_wait_us":3,"thread_start_us":13601,"threads_started":1}
I20260808 23:40:22.790495 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.761873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.18.186.129:45533)
0808 23:40:22.764740 (+ 2867us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.764754 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.764809 (+ 55us) server_negotiation.cc:407] Connection header received
0808 23:40:22.764903 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.764916 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.765282 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.765605 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.767875 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.768988 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.771147 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.772414 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.772555 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.779035 (+ 6480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.779052 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.779082 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.779151 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.785106 (+ 5955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.786180 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.786190 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.786194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.786469 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.787166 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.787171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.787174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.787772 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.789477 (+ 1705us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.789671 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.789988 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2598,"thread_start_us":2250,"threads_started":1}
I20260808 23:40:22.791013 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.742411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.18.186.190:42189)
0808 23:40:22.742653 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.742659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.756926 (+ 14267us) server_negotiation.cc:407] Connection header received
0808 23:40:22.757245 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.757252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.757362 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.757469 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.761326 (+ 3857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.762981 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.771990 (+ 9009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.773084 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.773217 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.780099 (+ 6882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.780107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.780111 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.780167 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.785161 (+ 4994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.786140 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.786146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.786149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.786363 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.787031 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.787036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.787039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.787505 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.787650 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.789993 (+ 2343us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.790286 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 23:40:22.792493 25999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:22.793365 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.742873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49350 (local address 127.18.186.189:39769)
0808 23:40:22.743080 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.743088 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.757162 (+ 14074us) server_negotiation.cc:407] Connection header received
0808 23:40:22.758702 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.758708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.758805 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.758896 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.770321 (+ 11425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.771547 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.779088 (+ 7541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.779938 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.780045 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.783996 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.784003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.784008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.784055 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.788860 (+ 4805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.790159 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.790165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.790169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.790370 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.790893 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.790898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.790902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.791414 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.791615 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.792604 (+ 989us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.792908 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:40:22.794051 26606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.742909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.0.0.1:49350)
0808 23:40:22.756997 (+ 14088us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.757016 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.757306 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.759178 (+ 1872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.759184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.768295 (+ 9111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.770120 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.770135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.771799 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.771804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.778891 (+ 7087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.778907 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.780547 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.780555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.780865 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.781785 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.781812 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.783754 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.789029 (+ 5275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.789033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.789037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.790018 (+ 981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.790531 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.790536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.790539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.790739 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.791734 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.791739 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.792086 (+ 347us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.792344 (+ 258us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.792953 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13892,"spinlock_wait_cycles":30464,"thread_start_us":13315,"threads_started":1}
I20260808 23:40:22.817993 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.757048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.18.186.188:39493)
0808 23:40:22.757203 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.757209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.774352 (+ 17143us) server_negotiation.cc:407] Connection header received
0808 23:40:22.774562 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.774567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.774637 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.774755 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.780102 (+ 5347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.781472 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.784255 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.785211 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.785296 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.803538 (+ 18242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.803552 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.803597 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.803648 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.812207 (+ 8559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.814161 (+ 1954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.814166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.814170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.814449 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.814930 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.814935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.814938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.815529 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.815705 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.817128 (+ 1423us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.817393 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":39}
I20260808 23:40:22.818526 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.756953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:39493 (local address 127.0.0.1:49100)
0808 23:40:22.774213 (+ 17260us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.774228 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.774441 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.774932 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.774937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.778294 (+ 3357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.779226 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.779241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.781918 (+ 2677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.781925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.784106 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.784120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.785747 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.785756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.786017 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.791592 (+ 5575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.791614 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.803162 (+ 11548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.812365 (+ 9203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.812370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.812385 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.814016 (+ 1631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.814602 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.814606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.814610 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.814807 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.815757 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.815761 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.816038 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.816218 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.816536 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17028,"thread_start_us":16717,"threads_started":1}
W20260808 23:40:22.819650 26532 tablet.cc:2369] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 23:40:22 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.190@KRBTEST.COM for oryx/127.18.186.188@KRBTEST.COM
I20260808 23:40:22.837852 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.808300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:39493 (local address 127.0.0.1:49108)
0808 23:40:22.808669 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.808689 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.808838 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.811157 (+ 2319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.811165 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.812078 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.812453 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.812471 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.821302 (+ 8831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.821312 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.823805 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.823829 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.825124 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.825133 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.825460 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.826444 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.826477 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.830363 (+ 3886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.834437 (+ 4074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.834443 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.834447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.834922 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.835498 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.835509 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.835512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.835700 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.836497 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.836502 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.836726 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.836892 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.837250 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235}
I20260808 23:40:22.838029 26610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.809096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.18.186.188:39493)
0808 23:40:22.810668 (+ 1572us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.810677 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.810731 (+ 54us) server_negotiation.cc:407] Connection header received
0808 23:40:22.810834 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.810843 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.810944 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.811069 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.812961 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.815356 (+ 2395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.824013 (+ 8657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.824935 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.825068 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.830622 (+ 5554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.830628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.830631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.830660 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.834289 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.835056 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.835065 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.835068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.835295 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.835821 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.835826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.835830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.836282 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.836566 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.837010 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.837233 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1360,"thread_start_us":1791,"threads_started":1}
I20260808 23:40:22.840054 25797 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.186.188:39493: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "efd512694f1846a0935af075037a343d"
member_type: NON_VOTER
last_known_addr {
host: "127.18.186.188"
port: 39493
}
attrs {
promote: true
}
}
dest_uuid: "b98e6c34670f4565a3da3894ac0502c9"
cas_config_opid_index: 5
I20260808 23:40:22.841077 25797 consensus_queue.cc:237] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } }
I20260808 23:40:22.842928 26411 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER efd512694f1846a0935af075037a343d (127.18.186.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.844044 26553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:22.849094 26553 raft_consensus.cc:1261] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Refusing update from remote peer b98e6c34670f4565a3da3894ac0502c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 23:40:22.854401 26611 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:22.855029 26411 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.186.188:39493
I20260808 23:40:22.855813 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.856420 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:22.868780 26464 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 LEARNER]: Refusing update from remote peer b98e6c34670f4565a3da3894ac0502c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 23:40:22.869683 26415 master.cc:537] Master@127.18.186.188:39493 shutting down...
I20260808 23:40:22.872218 26611 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 23:40:22.884795 26464 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER efd512694f1846a0935af075037a343d (127.18.186.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.889497 26496 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: NON_VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: true } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.889850 26496 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:40:22.893000 26611 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: attempting to promote NON_VOTER fb4bc7131cf44e9a87eea91b293def91 to VOTER
I20260808 23:40:22.894212 26611 consensus_queue.cc:237] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } }
I20260808 23:40:22.896869 26464 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 LEARNER]: Refusing update from remote peer b98e6c34670f4565a3da3894ac0502c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260808 23:40:22.897459 25781 consensus_peers.cc:487] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 -> Peer efd512694f1846a0935af075037a343d (127.18.186.188:39493): Couldn't send request to peer efd512694f1846a0935af075037a343d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 23:40:22.897552 26613 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260808 23:40:22.900527 26613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fb4bc7131cf44e9a87eea91b293def91 (127.18.186.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.903872 26464 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fb4bc7131cf44e9a87eea91b293def91 (127.18.186.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.904614 26611 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.904865 26611 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:22.905189 26611 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.905405 26611 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:22.908520 26496 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:22.908764 26496 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:40:22.914291 26415 raft_consensus.cc:2229] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:22.914790 26415 raft_consensus.cc:2258] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 23:40:22.915002 26588 raft_consensus.cc:659] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260808 23:40:22.915416 26415 tablet_replica.cc:330] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: stopping tablet replica
I20260808 23:40:22.924158 26415 master.cc:552] Master@127.18.186.188:39493 shutdown complete.
I20260808 23:40:22.924310 26415 master_runner.cc:305] Clearing existing system tablet
I20260808 23:40:22.924728 26415 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:22.924906 26415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:22.927989 26415 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260808 23:40:22.929718 26618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:22.930109 26415 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:22.930542 26415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 23:40:22.930714 26415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
uuid: "efd512694f1846a0935af075037a343d"
format_stamp: "Formatted at 2026-08-08 23:40:22 on dist-test-slave-5k54"
server_key: "dc75a76327c47c516457bb96f6d4890e"
server_key_iv: "54e65c44cddd231c5be0663076d7bee3"
server_key_version: "encryptionkey@0"
I20260808 23:40:22.935876 26415 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:22.942986 26415 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 23:40:22.943239 26415 log.cc:1196] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 23:40:22.944036 26415 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Deleting consensus metadata
I20260808 23:40:22.944398 26415 master_runner.cc:315] Copying system tablet from 127.18.186.190:42189
I20260808 23:40:22.947975 26415 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: tablet copy: Beginning tablet copy session from remote peer at address 127.18.186.190:42189
I20260808 23:40:22.970453 26624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.949087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:60350)
0808 23:40:22.952060 (+ 2973us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:22.952097 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:22.952393 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:22.952837 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:22.952842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:22.953769 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:22.954382 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.954402 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.956160 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.956165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.958150 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:22.958166 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.959427 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.959434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:22.959762 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.960679 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:22.960702 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:22.962213 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:22.966915 (+ 4702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.966920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.966926 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.967645 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.968235 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:22.968241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:22.968245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:22.968478 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:22.969359 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:22.969364 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:22.969613 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 23:40:22.969773 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 23:40:22.970041 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2807,"thread_start_us":2552,"threads_started":1}
I20260808 23:40:22.971716 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:22.949046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.18.186.190:42189)
0808 23:40:22.949271 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:22.949278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:22.952343 (+ 3065us) server_negotiation.cc:407] Connection header received
0808 23:40:22.952571 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:22.952579 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:22.952670 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:22.952781 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:22.954568 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.955931 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.958322 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:22.959199 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:22.959322 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:22.962472 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:22.962479 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:22.962483 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:22.962527 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:22.966707 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.967791 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.967797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.967801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.968081 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:22.968623 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:22.968630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:22.968634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:22.969223 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:22.969412 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:22.971032 (+ 1620us) server_negotiation.cc:299] Negotiation successful
0808 23:40:22.971229 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 23:40:22.972355 25819 tablet_copy_service.cc:140] P b98e6c34670f4565a3da3894ac0502c9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer efd512694f1846a0935af075037a343d ({username='oryx', principal='oryx/127.18.186.188@KRBTEST.COM'} at 127.0.0.1:60350)
I20260808 23:40:22.972539 25819 tablet_copy_service.cc:161] P b98e6c34670f4565a3da3894ac0502c9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer efd512694f1846a0935af075037a343d at {username='oryx', principal='oryx/127.18.186.188@KRBTEST.COM'} at 127.0.0.1:60350: session id = efd512694f1846a0935af075037a343d-00000000000000000000000000000000
I20260808 23:40:22.976344 25819 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 23:40:22.978353 26415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 23:40:22.978390 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 25867
I20260808 23:40:22.988385 26415 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: tablet copy: Starting download of 0 data blocks...
I20260808 23:40:22.988721 26415 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: tablet copy: Starting download of 1 WAL segments...
I20260808 23:40:22.991268 26415 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 23:40:22.995695 25819 tablet_copy_service.cc:342] P b98e6c34670f4565a3da3894ac0502c9: Request end of tablet copy session efd512694f1846a0935af075037a343d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.186.188@KRBTEST.COM'} at 127.0.0.1:60350
I20260808 23:40:22.995849 25819 tablet_copy_service.cc:433] P b98e6c34670f4565a3da3894ac0502c9: ending tablet copy session efd512694f1846a0935af075037a343d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer efd512694f1846a0935af075037a343d
I20260808 23:40:23.004084 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:45533
--local_ip_for_outbound_sockets=127.18.186.129
--tserver_master_addrs=127.18.186.190:42189
--webserver_port=35627
--webserver_interface=127.18.186.129
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab
--principal=oryx/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769,127.18.186.188:39493 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:40:23.008880 26415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:23.011687 26625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:23.012822 26626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:23.014353 26415 server_base.cc:1034] running on GCE node
W20260808 23:40:23.020152 26629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:23.020648 26415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:23.021339 26415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:23.022555 26415 hybrid_clock.cc:648] HybridClock initialized: now 1786232423022506 us; error 29 us; skew 500 ppm
I20260808 23:40:23.022799 26415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:23 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232423, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:23.025209 26415 init.cc:377] Logged in from keytab as oryx/127.18.186.188@KRBTEST.COM (short username oryx)
I20260808 23:40:23.028396 26415 webserver.cc:460] Webserver started at http://127.18.186.188:42927/ using document root <none> and password file <none>
I20260808 23:40:23.028759 26415 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:23.028864 26415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:23.032040 26415 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 23:40:23.033938 26636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:23.034384 26415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 23:40:23.034512 26415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
uuid: "efd512694f1846a0935af075037a343d"
format_stamp: "Formatted at 2026-08-08 23:40:22 on dist-test-slave-5k54"
server_key: "dc75a76327c47c516457bb96f6d4890e"
server_key_iv: "54e65c44cddd231c5be0663076d7bee3"
server_key_version: "encryptionkey@0"
I20260808 23:40:23.034711 26415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:23.046592 26415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:23.050666 26415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:23.076354 26415 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.188:39493
I20260808 23:40:23.076413 26691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.188:39493 every 8 connection(s)
I20260808 23:40:23.077390 26415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
I20260808 23:40:23.082631 26692 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:40:23.086680 26692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Bootstrap starting.
I20260808 23:40:23.106482 26692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:40:23.107029 26692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Bootstrap complete.
I20260808 23:40:23.108160 26692 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } }
I20260808 23:40:23.108470 26692 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: efd512694f1846a0935af075037a343d, State: Initialized, Role: LEARNER
I20260808 23:40:23.108762 26692 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } }
I20260808 23:40:23.109854 26692 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:23.109598 26695 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:23.110149 26695 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: This master's current role is: LEARNER
I20260808 23:40:23.118511 26707 catalog_manager.cc:1259] Loaded cluster ID: 79856681b2aa48eda54637c35a39f9e0
I20260808 23:40:23.118604 26707 catalog_manager.cc:1552] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: loading cluster ID for follower catalog manager: success
I20260808 23:40:23.121960 26707 catalog_manager.cc:1574] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: acquiring CA information for follower catalog manager: success
I20260808 23:40:23.124339 26707 catalog_manager.cc:1602] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:40:23.398559 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.389477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:39493 (local address 127.0.0.1:49122)
0808 23:40:23.391259 (+ 1782us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:23.391281 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:23.392355 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:23.392788 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:23.392793 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:23.392825 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:23.393061 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:23.393071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.394581 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.394590 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:23.396144 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:23.396157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.397411 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.397434 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:23.397753 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.397796 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 23:40:23.397927 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 23:40:23.398100 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1623,"thread_start_us":1514,"threads_started":1}
I20260808 23:40:23.398571 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.389729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.18.186.188:39493)
0808 23:40:23.391619 (+ 1890us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:23.391623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:23.391643 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:40:23.392549 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:23.392553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:23.392629 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:23.392762 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:23.393275 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.394439 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.396304 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.397217 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.397488 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.397712 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:23.398069 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 23:40:23.398200 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1786,"thread_start_us":1623,"threads_started":1}
I20260808 23:40:23.399724 26667 raft_consensus.cc:1203] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 23:40:23.400929 26695 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b98e6c34670f4565a3da3894ac0502c9. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:23.401402 26695 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: This master's current role is: LEARNER
W20260808 23:40:23.427096 26628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:23.427541 26628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:23.427771 26628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:23.438022 26628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:23.438144 26628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:23.438346 26628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:23.451031 26628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab
--principal=oryx/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:45533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=35627
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769,127.18.186.188:39493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:23.452734 26628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:23.454259 26628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:23.473922 26717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:23.473820 26719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:23.474288 26715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:23.475999 26628 server_base.cc:1034] running on GCE node
I20260808 23:40:23.476676 26628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:23.477731 26628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:23.478962 26628 hybrid_clock.cc:648] HybridClock initialized: now 1786232423478893 us; error 60 us; skew 500 ppm
I20260808 23:40:23.479310 26628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:23 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232423, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:23.483687 26628 init.cc:377] Logged in from keytab as oryx/127.18.186.129@KRBTEST.COM (short username oryx)
I20260808 23:40:23.485617 26628 webserver.cc:460] Webserver started at http://127.18.186.129:35627/ using document root <none> and password file <none>
I20260808 23:40:23.486413 26628 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:23.486550 26628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:23.492889 26628 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 23:40:23.496232 26726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:23.497690 26628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 23:40:23.497881 26628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "b2e55cc3d8604e7ea8b29d95071a837a"
format_stamp: "Formatted at 2026-08-08 23:40:20 on dist-test-slave-5k54"
server_key: "89f6acc6edfa960477b29a3e5d259a29"
server_key_iv: "a0e20b32859c0d2d8793f003c46b4a6b"
server_key_version: "encryptionkey@0"
I20260808 23:40:23.499136 26628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:23.516615 26628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:23.522460 26628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:23.522765 26628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:23.524390 26628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:23.526427 26628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:23.526599 26628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:23.526710 26628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:23.526793 26628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:23.604988 26628 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:45533
I20260808 23:40:23.605156 26850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:45533 every 8 connection(s)
I20260808 23:40:23.606935 26628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:23.616072 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 26628
I20260808 23:40:23.616622 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 26010
Aug 08 23:40:23 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232423, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.188@KRBTEST.COM
Aug 08 23:40:23 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232423, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.189@KRBTEST.COM
Aug 08 23:40:23 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.130@KRBTEST.COM
Aug 08 23:40:23 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232423, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:23.648092 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.610983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:33529 (local address 127.18.186.188:39493)
0808 23:40:23.611232 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:23.611238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:23.613820 (+ 2582us) server_negotiation.cc:407] Connection header received
0808 23:40:23.619147 (+ 5327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:23.619155 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:23.619250 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:23.619323 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:23.624170 (+ 4847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.625566 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.629189 (+ 3623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.630047 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.630157 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.638287 (+ 8130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:23.638294 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:23.638300 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:23.638352 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:23.643350 (+ 4998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:23.644479 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:23.644485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:23.644488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:23.644697 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:23.645396 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:23.645402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:23.645407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:23.645822 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:23.645980 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:23.647288 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0808 23:40:23.647620 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 23:40:23.649930 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.617925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.18.186.130:39299)
0808 23:40:23.619389 (+ 1464us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:23.619398 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:23.619448 (+ 50us) server_negotiation.cc:407] Connection header received
0808 23:40:23.619562 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:23.619573 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:23.619809 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:23.623358 (+ 3549us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:23.623884 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.626024 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.628449 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.629501 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.630657 (+ 1156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.638638 (+ 7981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:23.638654 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:23.638671 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:23.638714 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:23.644924 (+ 6210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:23.646138 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:23.646158 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:23.646161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:23.646442 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:23.646968 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:23.646973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:23.646976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:23.647543 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:23.647858 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:23.649115 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0808 23:40:23.649388 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1270,"thread_start_us":1041,"threads_started":1}
I20260808 23:40:23.650969 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.611816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:39493 (local address 127.18.186.129:33529)
0808 23:40:23.613668 (+ 1852us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:23.613702 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:23.618774 (+ 5072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:23.619868 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:23.619881 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:23.621738 (+ 1857us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:23.623883 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:23.623909 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.625757 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.625762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:23.629017 (+ 3255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:23.629033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.630715 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.630725 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:23.631282 (+ 557us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.632470 (+ 1188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:23.632504 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:23.637906 (+ 5402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:23.643595 (+ 5689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:23.643604 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:23.643624 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:23.644322 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:23.644938 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:23.644942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:23.644945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:23.645202 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:23.646082 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:23.646090 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:23.646811 (+ 721us) client_negotiation.cc:769] Sending connection context
0808 23:40:23.648415 (+ 1604us) client_negotiation.cc:240] Negotiation successful
0808 23:40:23.649241 (+ 826us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1449,"mutex_wait_us":376,"spinlock_wait_cycles":17920,"thread_start_us":965,"threads_started":1}
I20260808 23:40:23.652067 26145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:23.652705 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.610432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:60201 (local address 127.18.186.190:42189)
0808 23:40:23.611313 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:23.611331 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:23.613820 (+ 2489us) server_negotiation.cc:407] Connection header received
0808 23:40:23.619236 (+ 5416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:23.619241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:23.619333 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:23.619573 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:23.624106 (+ 4533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.625664 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.629194 (+ 3530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.630130 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.630238 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.642118 (+ 11880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:23.642125 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:23.642129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:23.642172 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:23.647655 (+ 5483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:23.649457 (+ 1802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:23.649466 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:23.649473 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:23.649685 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:23.650550 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:23.650556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:23.650560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:23.651041 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:23.651189 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:23.651993 (+ 804us) server_negotiation.cc:299] Negotiation successful
0808 23:40:23.652330 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":584,"threads_started":1}
I20260808 23:40:23.654059 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.611822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.129:60201)
0808 23:40:23.613644 (+ 1822us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:23.613696 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:23.618782 (+ 5086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:23.619867 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:23.619879 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:23.621735 (+ 1856us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:23.623879 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:23.623909 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.625916 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.625921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:23.628995 (+ 3074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:23.629011 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.631073 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.631083 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:23.631374 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.632649 (+ 1275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:23.632683 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:23.641858 (+ 9175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:23.647948 (+ 6090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:23.647956 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:23.647963 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:23.649261 (+ 1298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:23.649899 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:23.649907 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:23.649911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:23.650388 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:23.651254 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:23.651258 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:23.651535 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 23:40:23.652930 (+ 1395us) client_negotiation.cc:240] Negotiation successful
0808 23:40:23.653344 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1289,"mutex_wait_us":419,"spinlock_wait_cycles":126080,"thread_start_us":965,"threads_started":1}
I20260808 23:40:23.654106 26851 heartbeater.cc:347] Connected to a master server at 127.18.186.188:39493
I20260808 23:40:23.654615 26851 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:23.655014 26852 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42189
I20260808 23:40:23.655133 26852 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:23.656277 26852 heartbeater.cc:511] Master 127.18.186.190:42189 requested a full tablet report, sending...
I20260808 23:40:23.656605 26851 heartbeater.cc:511] Master 127.18.186.188:39493 requested a full tablet report, sending...
I20260808 23:40:23.658725 25797 ts_manager.cc:194] Re-registered known tserver with Master: b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533)
I20260808 23:40:23.660187 25797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:60201
I20260808 23:40:23.661540 26860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.614364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:60379 (local address 127.18.186.189:39769)
0808 23:40:23.615937 (+ 1573us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:23.615941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:23.616389 (+ 448us) server_negotiation.cc:407] Connection header received
0808 23:40:23.619710 (+ 3321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:23.619715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:23.619861 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:23.620189 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:23.624105 (+ 3916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.625666 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.629194 (+ 3528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.630148 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.630238 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.638210 (+ 7972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:23.638221 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:23.638227 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:23.638273 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:23.653115 (+ 14842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:23.658037 (+ 4922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:23.658044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:23.658047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:23.658279 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:23.659012 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:23.659017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:23.659020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:23.659579 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:23.659761 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:23.660545 (+ 784us) server_negotiation.cc:299] Negotiation successful
0808 23:40:23.660858 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1463,"thread_start_us":1340,"threads_started":1}
I20260808 23:40:23.661825 26657 ts_manager.cc:194] Registered new tserver with Master: b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533)
I20260808 23:40:23.664657 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:23.614151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.18.186.129:60379)
0808 23:40:23.616077 (+ 1926us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:23.616099 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:23.619514 (+ 3415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:23.620360 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:23.620370 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:23.621792 (+ 1422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:23.623879 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:23.623909 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.625922 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.625930 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:23.628995 (+ 3065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:23.629055 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:23.630732 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:23.630745 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:23.631281 (+ 536us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:23.632513 (+ 1232us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:23.632535 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:23.637860 (+ 5325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:23.653322 (+ 15462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:23.653330 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:23.653336 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:23.657859 (+ 4523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:23.658460 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:23.658468 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:23.658474 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:23.658870 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:23.659832 (+ 962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:23.659837 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:23.660007 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 23:40:23.660221 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 23:40:23.663789 (+ 3568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1794,"mutex_wait_us":311,"spinlock_wait_cycles":7296,"thread_start_us":2899,"threads_started":1}
I20260808 23:40:23.665753 26854 heartbeater.cc:347] Connected to a master server at 127.18.186.189:39769
I20260808 23:40:23.665943 26854 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:23.666805 26854 heartbeater.cc:511] Master 127.18.186.189:39769 requested a full tablet report, sending...
I20260808 23:40:23.669107 26454 ts_manager.cc:194] Registered new tserver with Master: b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533)
I20260808 23:40:23.751349 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 26010
I20260808 23:40:23.771526 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:39299
--local_ip_for_outbound_sockets=127.18.186.130
--tserver_master_addrs=127.18.186.190:42189
--webserver_port=40331
--webserver_interface=127.18.186.130
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab
--principal=oryx/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769,127.18.186.188:39493 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:40:23.956882 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: NON_VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: true } } }
I20260808 23:40:23.957340 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:23.958863 26612 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: attempting to promote NON_VOTER efd512694f1846a0935af075037a343d to VOTER
I20260808 23:40:23.959486 26612 consensus_queue.cc:237] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } }
I20260808 23:40:23.962073 26667 raft_consensus.cc:1261] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 LEARNER]: Refusing update from remote peer b98e6c34670f4565a3da3894ac0502c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 23:40:23.962105 26464 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Refusing update from remote peer b98e6c34670f4565a3da3894ac0502c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 23:40:23.962785 26612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 23:40:23.963302 26866 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 23:40:23.966562 26612 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, efd512694f1846a0935af075037a343d (127.18.186.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:23.970045 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:23.970314 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:23.970858 26464 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, efd512694f1846a0935af075037a343d (127.18.186.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:23.971650 26865 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:23.971961 26865 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:23.972608 26612 sys_catalog.cc:455] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:23.972888 26612 sys_catalog.cc:458] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9 [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:23.973502 26667 raft_consensus.cc:2932] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, efd512694f1846a0935af075037a343d (127.18.186.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:23.975879 26872 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:23.976293 26872 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:40:23.978847 26873 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b98e6c34670f4565a3da3894ac0502c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:23.979197 26873 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: This master's current role is: FOLLOWER
W20260808 23:40:24.142716 26864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:24.142999 26864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:24.143188 26864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:24.151016 26864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:24.151156 26864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:24.151357 26864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:24.161515 26864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab
--principal=oryx/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:39299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=40331
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769,127.18.186.188:39493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:24.162921 26864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:24.164356 26864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:24.178015 26878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:24.178081 26882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:24.178131 26879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:24.180855 26864 server_base.cc:1034] running on GCE node
I20260808 23:40:24.181468 26864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:24.182436 26864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:24.183643 26864 hybrid_clock.cc:648] HybridClock initialized: now 1786232424183588 us; error 50 us; skew 500 ppm
I20260808 23:40:24.184049 26864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232424, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:24.187305 26864 init.cc:377] Logged in from keytab as oryx/127.18.186.130@KRBTEST.COM (short username oryx)
I20260808 23:40:24.188831 26864 webserver.cc:460] Webserver started at http://127.18.186.130:40331/ using document root <none> and password file <none>
I20260808 23:40:24.189486 26864 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:24.189600 26864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:24.194802 26864 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:40:24.197549 26891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:24.199060 26864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:40:24.199194 26864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "69dc5d50059845138c68d58c469763e5"
format_stamp: "Formatted at 2026-08-08 23:40:20 on dist-test-slave-5k54"
server_key: "155052dad1e8e0d8dc05eaa4b232b9d8"
server_key_iv: "228974170093fdb25f67451a5dce759a"
server_key_version: "encryptionkey@0"
I20260808 23:40:24.200282 26864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:24.238035 26864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:24.242853 26864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:24.243141 26864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:24.244308 26864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:24.245733 26864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:24.245826 26864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:24.245913 26864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:24.245968 26864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:24.280289 26864 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:39299
I20260808 23:40:24.280462 27016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:39299 every 8 connection(s)
I20260808 23:40:24.281848 26864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:24.286480 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 26864
I20260808 23:40:24.286872 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 26155
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232424, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.188@KRBTEST.COM
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232424, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.189@KRBTEST.COM
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.131@KRBTEST.COM
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232424, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:24.324605 27024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.285036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:35923 (local address 127.18.186.188:39493)
0808 23:40:24.300292 (+ 15256us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.300301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.300329 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:40:24.302627 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.302632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.302703 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.302778 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.305817 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.307101 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.309810 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.310485 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.310588 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.315860 (+ 5272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.315866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.315872 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.315921 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.319744 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.320797 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.320803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.320806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.321018 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.321787 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.321794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.321798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.322353 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.322538 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.323790 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.324062 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15083,"thread_start_us":15290,"threads_started":1}
I20260808 23:40:24.327332 27025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.285578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:39493 (local address 127.18.186.130:35923)
0808 23:40:24.296928 (+ 11350us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:24.296998 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:24.302203 (+ 5205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:24.303205 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:24.303213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:24.304454 (+ 1241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:24.305604 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.305630 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.307318 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.307323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.309651 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.309666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.310982 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.310991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.311363 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.312650 (+ 1287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:24.312672 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:24.315591 (+ 2919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:24.319951 (+ 4360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.319961 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.319977 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.320656 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.321177 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.321185 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.321190 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.321616 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.322643 (+ 1027us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:24.322651 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:24.323170 (+ 519us) client_negotiation.cc:769] Sending connection context
0808 23:40:24.324930 (+ 1760us) client_negotiation.cc:240] Negotiation successful
0808 23:40:24.325591 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10931,"mutex_wait_us":251,"thread_start_us":13096,"threads_started":1}
I20260808 23:40:24.327792 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.300305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:49877 (local address 127.18.186.189:39769)
0808 23:40:24.300840 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.300847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.301426 (+ 579us) server_negotiation.cc:407] Connection header received
0808 23:40:24.302612 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.302621 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.302727 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.302867 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.305847 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.307413 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.310124 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.310690 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.310789 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.316261 (+ 5472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.316268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.316274 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.316342 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.322631 (+ 6289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.323817 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.323822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.323825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.324000 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.324500 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.324505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.324508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.325280 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.325481 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.326733 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.327021 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":239,"threads_started":1}
I20260808 23:40:24.328292 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.300762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.18.186.130:49877)
0808 23:40:24.301323 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:24.301347 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:24.302218 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:24.303209 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:24.303217 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:24.304410 (+ 1193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:24.305608 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.305636 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.307698 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.307706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.309936 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.309955 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.311126 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.311135 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.311352 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.312450 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:24.312489 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:24.316035 (+ 3546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:24.322771 (+ 6736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.322786 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.322789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.323671 (+ 882us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.324139 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.324145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.324149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.324363 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.325550 (+ 1187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:24.325555 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:24.325713 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 23:40:24.325906 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 23:40:24.326195 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":493,"spinlock_wait_cycles":16640,"thread_start_us":303,"threads_started":1}
I20260808 23:40:24.329610 27017 heartbeater.cc:347] Connected to a master server at 127.18.186.188:39493
I20260808 23:40:24.330045 27017 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:24.331364 27017 heartbeater.cc:511] Master 127.18.186.188:39493 requested a full tablet report, sending...
I20260808 23:40:24.332129 27021 heartbeater.cc:347] Connected to a master server at 127.18.186.189:39769
I20260808 23:40:24.332350 27021 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:24.333101 27021 heartbeater.cc:511] Master 127.18.186.189:39769 requested a full tablet report, sending...
I20260808 23:40:24.333765 26657 ts_manager.cc:194] Registered new tserver with Master: 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299)
I20260808 23:40:24.334565 26454 ts_manager.cc:194] Registered new tserver with Master: 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299)
I20260808 23:40:24.334797 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.284910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:53869 (local address 127.18.186.190:42189)
0808 23:40:24.310292 (+ 25382us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.310299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.310320 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:40:24.310396 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.310401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.310479 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.310591 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.311950 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.313084 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.315775 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.316421 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.316572 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.322466 (+ 5894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.322474 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.322480 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.322590 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.327976 (+ 5386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.329331 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.329337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.329340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.332180 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.332821 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.332826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.332829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.333275 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.333419 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.333998 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.334241 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25229,"thread_start_us":25123,"threads_started":1}
I20260808 23:40:24.334872 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.285578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.130:53869)
0808 23:40:24.300605 (+ 15027us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:24.300624 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:24.302203 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:24.310636 (+ 8433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:24.310641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:24.311318 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:24.311758 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.311794 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.313207 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.313213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.314856 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.314876 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.316610 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.316617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.316839 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.318077 (+ 1238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:24.318099 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:24.321562 (+ 3463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:24.328110 (+ 6548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.328115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.328135 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.328799 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.332360 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.332371 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.332373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.332683 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.333406 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:24.333411 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:24.333596 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 23:40:24.333817 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 23:40:24.334134 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14866,"mutex_wait_us":115,"spinlock_wait_cycles":25472,"thread_start_us":14388,"threads_started":1}
I20260808 23:40:24.335979 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.288466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.18.186.131:37883)
0808 23:40:24.303998 (+ 15532us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.304008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.305117 (+ 1109us) server_negotiation.cc:407] Connection header received
0808 23:40:24.305201 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.305208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.305442 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.305604 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.307993 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.309507 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.312486 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.313462 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.313600 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.321427 (+ 7827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.321443 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.321459 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.321501 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.327298 (+ 5797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.329892 (+ 2594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.329913 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.329916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.330251 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.330711 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.330715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.330718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.331299 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.331741 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.332965 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.333241 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15346,"thread_start_us":15204,"threads_started":1}
I20260808 23:40:24.336508 27019 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42189
I20260808 23:40:24.336710 27019 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:24.337599 27019 heartbeater.cc:511] Master 127.18.186.190:42189 requested a full tablet report, sending...
I20260808 23:40:24.337707 26290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:24.338697 25797 ts_manager.cc:194] Re-registered known tserver with Master: 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299)
I20260808 23:40:24.340147 25797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:53869
I20260808 23:40:24.422821 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 26155
I20260808 23:40:24.441453 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:37883
--local_ip_for_outbound_sockets=127.18.186.131
--tserver_master_addrs=127.18.186.190:42189
--webserver_port=32973
--webserver_interface=127.18.186.131
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab
--principal=oryx/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769,127.18.186.188:39493 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:40:24.664546 26852 heartbeater.cc:503] Master 127.18.186.190:42189 was elected leader, sending a full tablet report...
W20260808 23:40:24.784240 27032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:24.784636 27032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:24.784852 27032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:24.792119 27032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:24.792281 27032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:24.792479 27032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:24.802492 27032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab
--principal=oryx/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:37883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=32973
--tserver_master_addrs=127.18.186.190:42189,127.18.186.189:39769,127.18.186.188:39493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:24.803990 27032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:24.805474 27032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:24.819300 27042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:24.819309 27040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:24.819706 27039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:24.820158 27032 server_base.cc:1034] running on GCE node
I20260808 23:40:24.820804 27032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:24.821816 27032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:24.823076 27032 hybrid_clock.cc:648] HybridClock initialized: now 1786232424823029 us; error 39 us; skew 500 ppm
I20260808 23:40:24.823477 27032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232424, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:24.826874 27032 init.cc:377] Logged in from keytab as oryx/127.18.186.131@KRBTEST.COM (short username oryx)
I20260808 23:40:24.828445 27032 webserver.cc:460] Webserver started at http://127.18.186.131:32973/ using document root <none> and password file <none>
I20260808 23:40:24.829147 27032 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:24.829310 27032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:24.834810 27032 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:40:24.838027 27051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:24.839398 27032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:40:24.839663 27032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "dfa6b0c648d64bbe8675e8f4cc891256"
format_stamp: "Formatted at 2026-08-08 23:40:21 on dist-test-slave-5k54"
server_key: "03ba9ed4d1350bc79f63fe079df6ac8f"
server_key_iv: "f980aded583f43d478d3e625a82134de"
server_key_version: "encryptionkey@0"
I20260808 23:40:24.840736 27032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:24.863965 27032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:24.868824 27032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:24.869166 27032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:24.870694 27032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:24.875277 27032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:24.875463 27032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:24.875586 27032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:24.875674 27032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:24.913849 27032 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:37883
I20260808 23:40:24.913971 27174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:37883 every 8 connection(s)
I20260808 23:40:24.915341 27032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:24.917114 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 27032
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232424, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.189@KRBTEST.COM
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232424, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:24.969861 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.919274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:48481 (local address 127.18.186.190:42189)
0808 23:40:24.929653 (+ 10379us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.929664 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.932351 (+ 2687us) server_negotiation.cc:407] Connection header received
0808 23:40:24.937230 (+ 4879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.937240 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.937354 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.937510 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.941473 (+ 3963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.943069 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.948169 (+ 5100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.949227 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.949408 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.958033 (+ 8625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.958070 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.958076 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.958128 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.963769 (+ 5641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.964921 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.964931 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.964934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.965104 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.966964 (+ 1860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.966971 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.966976 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.967513 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.967647 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.968671 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.968920 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10238,"thread_start_us":10125,"threads_started":1}
I20260808 23:40:24.970290 27179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.918483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:50589 (local address 127.18.186.189:39769)
0808 23:40:24.931674 (+ 13191us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.931684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.936115 (+ 4431us) server_negotiation.cc:407] Connection header received
0808 23:40:24.937039 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.937049 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.937191 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.937291 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.941512 (+ 4221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.942907 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.948268 (+ 5361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.949052 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.949175 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.956780 (+ 7605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.956788 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.956794 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.956857 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.962275 (+ 5418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.963771 (+ 1496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.963777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.963781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.964078 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.966866 (+ 2788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.966871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.966874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.967328 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.967493 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.969365 (+ 1872us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.969675 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13010,"thread_start_us":13202,"threads_started":1}
I20260808 23:40:24.971113 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.919267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.131:48481)
0808 23:40:24.932142 (+ 12875us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:24.932195 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:24.936383 (+ 4188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:24.940408 (+ 4025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:24.940414 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:24.940957 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:24.941285 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.941300 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.945892 (+ 4592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.945900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.947688 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.947706 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.951145 (+ 3439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.951156 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.951403 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.952364 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:24.952389 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:24.957736 (+ 5347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:24.963992 (+ 6256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.963997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.964001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.964756 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.965232 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.965240 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.965243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.966757 (+ 1514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.967769 (+ 1012us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:24.967784 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:24.968208 (+ 424us) client_negotiation.cc:769] Sending connection context
0808 23:40:24.968537 (+ 329us) client_negotiation.cc:240] Negotiation successful
0808 23:40:24.969253 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12520,"mutex_wait_us":40,"spinlock_wait_cycles":19328,"thread_start_us":16147,"threads_started":1}
I20260808 23:40:24.973809 27176 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42189
I20260808 23:40:24.974184 27182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.919923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.18.186.131:50589)
0808 23:40:24.935964 (+ 16041us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:24.935991 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:24.936851 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:24.937872 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:24.937885 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:24.939011 (+ 1126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:24.940129 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.940183 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.943154 (+ 2971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.943162 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.945724 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.945743 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.949605 (+ 3862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.949625 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.950014 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.951328 (+ 1314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:24.951370 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:24.956478 (+ 5108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:24.962499 (+ 6021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.962519 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.962538 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.963249 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.964249 (+ 1000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.964253 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.964257 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.966657 (+ 2400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.967590 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:24.967599 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:24.968216 (+ 617us) client_negotiation.cc:769] Sending connection context
0808 23:40:24.972525 (+ 4309us) client_negotiation.cc:240] Negotiation successful
0808 23:40:24.972866 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15866,"mutex_wait_us":2,"spinlock_wait_cycles":18304,"thread_start_us":11749,"threads_started":1}
I20260808 23:40:24.974357 27176 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:24.975379 27177 heartbeater.cc:347] Connected to a master server at 127.18.186.189:39769
I20260808 23:40:24.975608 27177 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:24.976547 27176 heartbeater.cc:511] Master 127.18.186.190:42189 requested a full tablet report, sending...
I20260808 23:40:24.978677 25797 ts_manager.cc:194] Re-registered known tserver with Master: dfa6b0c648d64bbe8675e8f4cc891256 (127.18.186.131:37883)
I20260808 23:40:24.979465 27177 heartbeater.cc:511] Master 127.18.186.189:39769 requested a full tablet report, sending...
I20260808 23:40:24.980096 25797 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:48481
I20260808 23:40:24.982128 26454 ts_manager.cc:194] Registered new tserver with Master: dfa6b0c648d64bbe8675e8f4cc891256 (127.18.186.131:37883)
Aug 08 23:40:24 dist-test-slave-5k54 krb5kdc[25748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232424, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.188@KRBTEST.COM
I20260808 23:40:24.992417 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.948130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.18.186.188:39493)
0808 23:40:24.953752 (+ 5622us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.953759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.953778 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:40:24.953835 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.953844 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.953959 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.955129 (+ 1170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.961351 (+ 6222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.965156 (+ 3805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.970528 (+ 5372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.971277 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.971441 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.977737 (+ 6296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.977743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.977748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.977791 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.986871 (+ 9080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.987974 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.987980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.987984 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.988168 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.989268 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.989277 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.989280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.989768 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.990234 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.991533 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.991856 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5480,"thread_start_us":5319,"threads_started":1}
I20260808 23:40:24.993507 27189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.935224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35092 (local address 127.18.186.190:42189)
0808 23:40:24.950314 (+ 15090us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.950324 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.950349 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:40:24.950434 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.950443 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.950544 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.950689 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.959174 (+ 8485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.961046 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.966336 (+ 5290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.967475 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.967614 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.979956 (+ 12342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.979964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.979969 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.980026 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.986304 (+ 6278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.989340 (+ 3036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.989348 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.989354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.989632 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.990287 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.990295 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.990298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.990799 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.991156 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.992513 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.992797 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14913,"thread_start_us":14707,"threads_started":1}
I20260808 23:40:24.995249 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.937877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:39493 (local address 127.18.186.131:47251)
0808 23:40:24.949926 (+ 12049us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:24.949953 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:24.950072 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:24.953096 (+ 3024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:24.953101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:24.955781 (+ 2680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:24.957302 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.957319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.964094 (+ 6775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.964101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.968418 (+ 4317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:24.968432 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.974493 (+ 6061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.974501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:24.975044 (+ 543us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.976545 (+ 1501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:24.976573 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:24.985710 (+ 9137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:24.991638 (+ 5928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.991643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.991647 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.992311 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.992859 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:24.992865 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:24.992869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:24.993120 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:24.993996 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:24.994001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:24.994206 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 23:40:24.994389 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 23:40:24.994731 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11918,"thread_start_us":12266,"threads_started":1}
I20260808 23:40:24.996248 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.937813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.18.186.189:39769)
0808 23:40:24.947451 (+ 9638us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.947457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.951808 (+ 4351us) server_negotiation.cc:407] Connection header received
0808 23:40:24.951868 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.951876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.951955 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.952060 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.961595 (+ 9535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.963370 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.976787 (+ 13417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.977628 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.977763 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.985855 (+ 8092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.985867 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.985874 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.985918 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.991154 (+ 5236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.992329 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.992351 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.992355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.992902 (+ 547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.993510 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.993516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.993519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.994036 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.994429 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.995188 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.995504 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9483,"thread_start_us":9367,"threads_started":1}
I20260808 23:40:24.996380 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:24.933089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:47251 (local address 127.18.186.188:39493)
0808 23:40:24.948154 (+ 15065us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:24.948159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:24.950469 (+ 2310us) server_negotiation.cc:407] Connection header received
0808 23:40:24.950546 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:24.950552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:24.950658 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:24.950802 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:24.957585 (+ 6783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.963851 (+ 6266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.972770 (+ 8919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:24.973910 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:24.974026 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:24.986131 (+ 12105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:24.986142 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:24.986148 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:24.986194 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:24.991455 (+ 5261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.992452 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.992457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.992460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.992692 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:24.993291 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:24.993297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:24.993300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:24.993726 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:24.993921 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:24.995632 (+ 1711us) server_negotiation.cc:299] Negotiation successful
0808 23:40:24.995875 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14954,"thread_start_us":14853,"threads_started":1}
I20260808 23:40:24.997196 27175 heartbeater.cc:347] Connected to a master server at 127.18.186.188:39493
I20260808 23:40:24.997357 27175 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:24.997869 27175 heartbeater.cc:511] Master 127.18.186.188:39493 requested a full tablet report, sending...
I20260808 23:40:24.999226 26657 ts_manager.cc:194] Registered new tserver with Master: dfa6b0c648d64bbe8675e8f4cc891256 (127.18.186.131:37883)
I20260808 23:40:25.000174 25797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:40:25.009488 25797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:25.058585 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.036241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.18.186.130:39299)
0808 23:40:25.046219 (+ 9978us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.046232 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.048481 (+ 2249us) server_negotiation.cc:407] Connection header received
0808 23:40:25.048729 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.048745 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.049009 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.049266 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:25.050655 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.052649 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.054960 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.056897 (+ 1937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.057095 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.057331 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.057923 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.058176 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9778,"thread_start_us":9615,"threads_started":1}
I20260808 23:40:25.058638 27209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.036038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:39299 (local address 127.0.0.1:38678)
0808 23:40:25.048339 (+ 12301us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:25.048356 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:25.048548 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:25.049245 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:25.049249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:25.049278 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:25.050201 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.050248 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.052856 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.052862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.054805 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.054818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.057526 (+ 2708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.057532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.057725 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.057749 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:40:25.057875 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 23:40:25.058026 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12171,"thread_start_us":11884,"threads_started":1}
I20260808 23:40:25.063923 26944 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0d0614225e437ca31aeda07f8267a5 (DEFAULT_TABLE table=test-table [id=ac9876eee1284b1d87135831e918ebf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:25.066072 26944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0d0614225e437ca31aeda07f8267a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:25.068087 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.036619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.18.186.131:37883)
0808 23:40:25.052454 (+ 15835us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.052464 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.056192 (+ 3728us) server_negotiation.cc:407] Connection header received
0808 23:40:25.056336 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.056345 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.056634 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.056914 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:25.060089 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.062476 (+ 2387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.065520 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.066722 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.066945 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.067149 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.067497 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.067668 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15586,"thread_start_us":15313,"threads_started":1}
I20260808 23:40:25.068135 27210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.036218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37883 (local address 127.0.0.1:57060)
0808 23:40:25.055851 (+ 19633us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:25.055871 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:25.056039 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:25.059094 (+ 3055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:25.059100 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:25.059126 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:25.059790 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.059814 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.062638 (+ 2824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.062644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.065376 (+ 2732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.065388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.067016 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.067024 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.067258 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.067283 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:40:25.067396 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 23:40:25.067551 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19473,"mutex_wait_us":7,"thread_start_us":18924,"threads_started":1}
I20260808 23:40:25.072216 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.034975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45533 (local address 127.0.0.1:48442)
0808 23:40:25.042395 (+ 7420us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:25.042436 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:25.042602 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:25.061232 (+ 18630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:25.061237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:25.061263 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:25.061942 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.061952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.064453 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.064460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.067869 (+ 3409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.067884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.070931 (+ 3047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.070938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.071191 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.071224 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:40:25.071325 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 23:40:25.071477 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7214,"mutex_wait_us":1,"thread_start_us":832,"threads_started":1}
I20260808 23:40:25.072016 27104 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0d0614225e437ca31aeda07f8267a5 (DEFAULT_TABLE table=test-table [id=ac9876eee1284b1d87135831e918ebf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:25.073086 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.035364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.18.186.129:45533)
0808 23:40:25.060674 (+ 25310us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.060681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.060705 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:40:25.060787 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.060792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.061023 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.061252 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:25.062240 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.064274 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.068573 (+ 4299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.070591 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.070884 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.071107 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.071389 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.071649 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25103,"thread_start_us":37018,"threads_started":1}
I20260808 23:40:25.073896 27104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0d0614225e437ca31aeda07f8267a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:25.079906 26777 tablet_service.cc:1467] Processing CreateTablet for tablet 6f0d0614225e437ca31aeda07f8267a5 (DEFAULT_TABLE table=test-table [id=ac9876eee1284b1d87135831e918ebf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:25.081617 26777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f0d0614225e437ca31aeda07f8267a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:25.099586 27214 tablet_bootstrap.cc:492] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Bootstrap starting.
I20260808 23:40:25.106658 27214 tablet_bootstrap.cc:654] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:25.108457 27215 tablet_bootstrap.cc:492] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Bootstrap starting.
I20260808 23:40:25.112764 27213 tablet_bootstrap.cc:492] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Bootstrap starting.
I20260808 23:40:25.115412 27215 tablet_bootstrap.cc:654] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:25.115365 27214 log.cc:824] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:25.118134 27215 log.cc:824] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:25.118180 27213 tablet_bootstrap.cc:654] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:25.120261 27213 log.cc:824] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:25.121691 27214 tablet_bootstrap.cc:492] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: No bootstrap required, opened a new log
I20260808 23:40:25.122079 27214 ts_tablet_manager.cc:1397] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.010s
I20260808 23:40:25.124925 27215 tablet_bootstrap.cc:492] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: No bootstrap required, opened a new log
I20260808 23:40:25.125340 27215 ts_tablet_manager.cc:1397] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260808 23:40:25.127153 27213 tablet_bootstrap.cc:492] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: No bootstrap required, opened a new log
I20260808 23:40:25.127542 27213 ts_tablet_manager.cc:1397] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.008s
I20260808 23:40:25.134289 27214 raft_consensus.cc:348] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.134979 27214 raft_consensus.cc:374] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:25.135177 27214 raft_consensus.cc:729] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa6b0c648d64bbe8675e8f4cc891256, State: Initialized, Role: FOLLOWER
I20260808 23:40:25.136176 27214 consensus_queue.cc:260] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.138824 27214 ts_tablet_manager.cc:1428] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Time spent starting tablet: real 0.017s user 0.009s sys 0.008s
I20260808 23:40:25.139154 27176 heartbeater.cc:503] Master 127.18.186.190:42189 was elected leader, sending a full tablet report...
I20260808 23:40:25.140764 27215 raft_consensus.cc:348] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.141499 27215 raft_consensus.cc:374] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:25.141676 27215 raft_consensus.cc:729] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e55cc3d8604e7ea8b29d95071a837a, State: Initialized, Role: FOLLOWER
I20260808 23:40:25.142467 27215 consensus_queue.cc:260] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.144296 27213 raft_consensus.cc:348] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.144959 27213 raft_consensus.cc:374] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:25.145141 27213 raft_consensus.cc:729] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69dc5d50059845138c68d58c469763e5, State: Initialized, Role: FOLLOWER
I20260808 23:40:25.145712 27215 ts_tablet_manager.cc:1428] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Time spent starting tablet: real 0.020s user 0.009s sys 0.011s
I20260808 23:40:25.146018 27213 consensus_queue.cc:260] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.154031 27019 heartbeater.cc:503] Master 127.18.186.190:42189 was elected leader, sending a full tablet report...
I20260808 23:40:25.154651 27213 ts_tablet_manager.cc:1428] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Time spent starting tablet: real 0.027s user 0.013s sys 0.011s
W20260808 23:40:25.174187 27178 tablet.cc:2369] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:40:25.285574 27022 tablet.cc:2369] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:40:25.363623 26855 tablet.cc:2369] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:40:25.365116 27219 raft_consensus.cc:482] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:25.365479 27219 raft_consensus.cc:504] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.367600 27219 leader_election.cc:288] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533), 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299)
I20260808 23:40:25.376472 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.368547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:33589 (local address 127.18.186.130:39299)
0808 23:40:25.368743 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.368747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.368763 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:40:25.368824 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.368827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.368882 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.368991 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:25.369609 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.370428 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.373732 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.375393 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.375599 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.375683 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.375980 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.376125 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:40:25.376489 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.367856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:39299 (local address 127.18.186.131:33589)
0808 23:40:25.368168 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:25.368201 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:25.368364 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:25.369011 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:25.369015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:25.369039 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:25.369418 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.369428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.370623 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.370627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.373536 (+ 2909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.373552 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.375628 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.375633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.375788 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.375813 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:40:25.375954 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 23:40:25.376104 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":21}
I20260808 23:40:25.376976 27208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.368224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:58047 (local address 127.18.186.129:45533)
0808 23:40:25.368526 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.368533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.368666 (+ 133us) server_negotiation.cc:407] Connection header received
0808 23:40:25.368863 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.368867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.368926 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.369025 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:25.369649 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.370955 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.374031 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.375835 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.376041 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.376112 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.376464 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.376637 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:40:25.377139 27182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.368094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45533 (local address 127.18.186.131:58047)
0808 23:40:25.368571 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:25.368593 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:25.368719 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:25.369046 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:25.369051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:25.369090 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:25.369463 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.369472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.371125 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.371143 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.373805 (+ 2662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:25.373815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.375985 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.375990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:25.376246 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.376292 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 23:40:25.376385 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 23:40:25.376712 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":2,"spinlock_wait_cycles":4992}
I20260808 23:40:25.377930 26968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0d0614225e437ca31aeda07f8267a5" candidate_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69dc5d50059845138c68d58c469763e5" is_pre_election: true
I20260808 23:40:25.378902 26968 raft_consensus.cc:2454] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa6b0c648d64bbe8675e8f4cc891256 in term 0.
I20260808 23:40:25.378829 26801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0d0614225e437ca31aeda07f8267a5" candidate_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" is_pre_election: true
I20260808 23:40:25.379865 27054 leader_election.cc:302] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69dc5d50059845138c68d58c469763e5, dfa6b0c648d64bbe8675e8f4cc891256; no voters:
I20260808 23:40:25.379819 26801 raft_consensus.cc:2454] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa6b0c648d64bbe8675e8f4cc891256 in term 0.
I20260808 23:40:25.380564 27219 raft_consensus.cc:2781] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:25.380950 27219 raft_consensus.cc:482] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:25.381245 27219 raft_consensus.cc:3037] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:25.384709 27219 raft_consensus.cc:504] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.385603 27219 leader_election.cc:288] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [CANDIDATE]: Term 1 election: Requested vote from peers b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533), 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299)
I20260808 23:40:25.386157 26801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0d0614225e437ca31aeda07f8267a5" candidate_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e55cc3d8604e7ea8b29d95071a837a"
I20260808 23:40:25.386212 26968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f0d0614225e437ca31aeda07f8267a5" candidate_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69dc5d50059845138c68d58c469763e5"
I20260808 23:40:25.386381 26801 raft_consensus.cc:3037] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:25.386487 26968 raft_consensus.cc:3037] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:25.389724 26968 raft_consensus.cc:2454] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa6b0c648d64bbe8675e8f4cc891256 in term 1.
I20260808 23:40:25.389837 26801 raft_consensus.cc:2454] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa6b0c648d64bbe8675e8f4cc891256 in term 1.
I20260808 23:40:25.390328 27054 leader_election.cc:302] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69dc5d50059845138c68d58c469763e5, dfa6b0c648d64bbe8675e8f4cc891256; no voters:
I20260808 23:40:25.390748 27219 raft_consensus.cc:2781] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:25.391319 27219 raft_consensus.cc:686] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 LEADER]: Becoming Leader. State: Replica: dfa6b0c648d64bbe8675e8f4cc891256, State: Running, Role: LEADER
I20260808 23:40:25.392172 27219 consensus_queue.cc:237] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } }
I20260808 23:40:25.398451 25797 catalog_manager.cc:5495] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfa6b0c648d64bbe8675e8f4cc891256 (127.18.186.131). New cstate: current_term: 1 leader_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } health_report { overall_health: UNKNOWN } } }
I20260808 23:40:25.426731 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.420062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.18.186.131:37883)
0808 23:40:25.420318 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.420325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.420589 (+ 264us) server_negotiation.cc:407] Connection header received
0808 23:40:25.420753 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.420757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.420839 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.420982 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:25.421359 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.422132 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.424006 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.424712 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.424827 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.425469 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:25.425992 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:25.426217 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.426281 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.426368 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:40:25.437256 26968 raft_consensus.cc:1261] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 1 FOLLOWER]: Refusing update from remote peer dfa6b0c648d64bbe8675e8f4cc891256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:25.437393 26801 raft_consensus.cc:1261] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 1 FOLLOWER]: Refusing update from remote peer dfa6b0c648d64bbe8675e8f4cc891256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:25.438511 27219 consensus_queue.cc:1035] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:25.438952 27223 consensus_queue.cc:1035] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:25.458477 27229 mvcc.cc:205] Tried to move back new op lower bound from 7316408014579314688 to 7316408014410522624. Current Snapshot: MvccSnapshot[applied={T|T < 7316408014579314688}]
I20260808 23:40:25.465142 27228 mvcc.cc:205] Tried to move back new op lower bound from 7316408014579314688 to 7316408014410522624. Current Snapshot: MvccSnapshot[applied={T|T < 7316408014579314688}]
I20260808 23:40:25.485457 25797 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:25.485757 25797 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:25.493469 25797 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b98e6c34670f4565a3da3894ac0502c9: Sending DeleteTablet for 3 replicas of tablet 6f0d0614225e437ca31aeda07f8267a5
I20260808 23:40:25.494933 26777 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f0d0614225e437ca31aeda07f8267a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:25 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:48442
I20260808 23:40:25.496172 27239 tablet_replica.cc:330] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: stopping tablet replica
I20260808 23:40:25.496701 27104 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f0d0614225e437ca31aeda07f8267a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:25 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:57060
I20260808 23:40:25.496749 26944 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f0d0614225e437ca31aeda07f8267a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:25 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:38678
I20260808 23:40:25.497972 27239 raft_consensus.cc:2229] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:25.498720 27239 raft_consensus.cc:2258] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:25.501526 27239 ts_tablet_manager.cc:1905] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:25.504534 27240 tablet_replica.cc:330] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: stopping tablet replica
I20260808 23:40:25.505280 27240 raft_consensus.cc:2229] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:25.506125 27240 raft_consensus.cc:2258] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:25.509321 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 25762
I20260808 23:40:25.510686 25831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:25.516705 27239 ts_tablet_manager.cc:1918] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:25.517225 27239 log.cc:1196] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/6f0d0614225e437ca31aeda07f8267a5
I20260808 23:40:25.519265 27239 ts_tablet_manager.cc:1939] T 6f0d0614225e437ca31aeda07f8267a5 P b2e55cc3d8604e7ea8b29d95071a837a: Deleting consensus metadata
I20260808 23:40:25.519632 27240 ts_tablet_manager.cc:1905] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:25.529807 27240 ts_tablet_manager.cc:1918] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:25.530062 27240 log.cc:1196] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/6f0d0614225e437ca31aeda07f8267a5
I20260808 23:40:25.530500 27241 tablet_replica.cc:330] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: stopping tablet replica
I20260808 23:40:25.531047 27240 ts_tablet_manager.cc:1939] T 6f0d0614225e437ca31aeda07f8267a5 P 69dc5d50059845138c68d58c469763e5: Deleting consensus metadata
I20260808 23:40:25.531153 27241 raft_consensus.cc:2229] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:40:25.533622 27241 raft_consensus.cc:2258] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:25.537715 27241 ts_tablet_manager.cc:1905] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:25.554147 27241 ts_tablet_manager.cc:1918] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:25.554437 27241 log.cc:1196] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/6f0d0614225e437ca31aeda07f8267a5
I20260808 23:40:25.555231 27241 ts_tablet_manager.cc:1939] T 6f0d0614225e437ca31aeda07f8267a5 P dfa6b0c648d64bbe8675e8f4cc891256: Deleting consensus metadata
I20260808 23:40:25.555851 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.500207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.18.186.189:39769)
0808 23:40:25.528867 (+ 28660us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.528874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.528897 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:40:25.528968 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.528973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.529062 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.529210 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:25.532000 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.533262 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.535851 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.536567 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.536709 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.542258 (+ 5549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:25.542265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:25.542270 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:25.542321 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:25.547971 (+ 5650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:25.549268 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:25.549274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:25.549279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:25.549498 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:25.550128 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:25.550134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:25.550137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:25.550578 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:25.550893 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.551278 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.551632 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28525,"thread_start_us":29152,"threads_started":1}
I20260808 23:40:25.577728 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.505222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.18.186.188:39493)
0808 23:40:25.528301 (+ 23079us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.528309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.552265 (+ 23956us) server_negotiation.cc:407] Connection header received
0808 23:40:25.552450 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.552456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.552552 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.552664 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:25.557710 (+ 5046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.559165 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.561655 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.562686 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.562819 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.568282 (+ 5463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:25.568290 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:25.568295 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:25.568370 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:25.573888 (+ 5518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:25.574911 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:25.574917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:25.574932 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:25.575121 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:25.575699 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:25.575706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:25.575710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:25.576221 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:25.576554 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.576986 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.577266 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22910,"thread_start_us":23086,"threads_started":1}
I20260808 23:40:25.692354 25783 catalog_manager.cc:4841] TS 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299): tablet 6f0d0614225e437ca31aeda07f8267a5 (table test-table [id=ac9876eee1284b1d87135831e918ebf9]) successfully deleted
I20260808 23:40:25.693178 25782 catalog_manager.cc:4841] TS b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533): tablet 6f0d0614225e437ca31aeda07f8267a5 (table test-table [id=ac9876eee1284b1d87135831e918ebf9]) successfully deleted
I20260808 23:40:25.693256 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 25762
W20260808 23:40:25.714455 27052 connection.cc:535] server connection from 127.0.0.1:57060 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 23:40:25.736891 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.721296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.18.186.188:39493)
0808 23:40:25.721483 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.721488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.721503 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:40:25.721719 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.721726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.721822 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.721933 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:25.724206 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.725473 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.727290 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.727945 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.728088 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.730986 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:25.730992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:25.730997 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:25.731040 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:25.733838 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:25.734471 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:25.734477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:25.734480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:25.734642 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:25.735087 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:25.735092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:25.735096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:25.735526 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:25.735777 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.736253 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.736492 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 23:40:25.739300 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:25.719884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.18.186.189:39769)
0808 23:40:25.720149 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:25.720159 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:25.721275 (+ 1116us) server_negotiation.cc:407] Connection header received
0808 23:40:25.721822 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:25.721828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:25.721919 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:25.722017 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:25.724391 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.725146 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.727074 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:25.727892 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:25.728021 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:25.732241 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:25.732247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:25.732251 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:25.732299 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:25.736371 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:25.737168 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:25.737173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:25.737176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:25.737335 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:25.737721 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:25.737727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:25.737730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:25.738000 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:25.738260 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:25.738899 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 23:40:25.739090 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 23:40:26.471156 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:26.470154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.131:46141)
0808 23:40:26.470712 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:26.470888 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":257,"threads_started":1}
I20260808 23:40:26.471587 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:26.470481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.130:51647)
0808 23:40:26.470974 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:26.471122 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":225,"threads_started":1}
W20260808 23:40:26.471719 27176 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:42189 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
W20260808 23:40:26.472072 27019 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:42189 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
I20260808 23:40:26.477058 27256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:26.476163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.129:41069)
0808 23:40:26.476681 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:26.476851 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":288,"threads_started":1}
W20260808 23:40:26.477512 26852 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:42189 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
I20260808 23:40:26.992992 27257 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b98e6c34670f4565a3da3894ac0502c9)
I20260808 23:40:26.993280 27257 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } }
I20260808 23:40:26.994642 27258 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b98e6c34670f4565a3da3894ac0502c9)
I20260808 23:40:26.994774 27257 leader_election.cc:288] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b98e6c34670f4565a3da3894ac0502c9 (127.18.186.190:42189), fb4bc7131cf44e9a87eea91b293def91 (127.18.186.189:39769)
I20260808 23:40:26.994952 27258 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } }
I20260808 23:40:26.996213 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:26.995019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35478)
0808 23:40:26.995676 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:26.995894 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":393,"threads_started":1}
W20260808 23:40:26.996587 26641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
I20260808 23:40:27.003022 27258 leader_election.cc:288] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b98e6c34670f4565a3da3894ac0502c9 (127.18.186.190:42189), efd512694f1846a0935af075037a343d (127.18.186.188:39493)
I20260808 23:40:27.006567 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:26.995324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39944 (local address 127.18.186.189:39769)
0808 23:40:26.995995 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:26.996001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:26.996568 (+ 567us) server_negotiation.cc:407] Connection header received
0808 23:40:26.996870 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:26.996879 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:26.996971 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:26.997069 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:26.998079 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:26.999729 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.003046 (+ 3317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.004851 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.005023 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.005137 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.005886 (+ 749us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.006096 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":329,"threads_started":1}
I20260808 23:40:27.007443 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:26.995072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:39769 (local address 127.0.0.1:39944)
0808 23:40:26.996461 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:26.996478 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:26.996689 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:26.997379 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:26.997387 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:26.997437 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:26.997870 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:26.997884 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:26.999979 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:26.999987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.002800 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.002821 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.005353 (+ 2532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.005362 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.005647 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.005700 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 23:40:27.006740 (+ 1040us) client_negotiation.cc:240] Negotiation successful
0808 23:40:27.006933 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1256,"thread_start_us":1092,"threads_started":1}
I20260808 23:40:27.008323 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:26.996770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35484)
0808 23:40:27.007491 (+ 10721us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.007988 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10552,"threads_started":1}
I20260808 23:40:27.008476 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.008104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35494)
0808 23:40:27.008218 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.008312 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 23:40:27.008697 26641 leader_election.cc:334] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b98e6c34670f4565a3da3894ac0502c9 (127.18.186.190:42189): Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
W20260808 23:40:27.008806 26439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
I20260808 23:40:27.010295 26464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efd512694f1846a0935af075037a343d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fb4bc7131cf44e9a87eea91b293def91" is_pre_election: true
I20260808 23:40:27.010763 26464 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd512694f1846a0935af075037a343d in term 1.
I20260808 23:40:27.011111 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.010415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35502)
0808 23:40:27.010647 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.010859 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260808 23:40:27.011353 26439 leader_election.cc:334] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b98e6c34670f4565a3da3894ac0502c9 (127.18.186.190:42189): Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
I20260808 23:40:27.011338 26637 leader_election.cc:302] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: efd512694f1846a0935af075037a343d, fb4bc7131cf44e9a87eea91b293def91; no voters: b98e6c34670f4565a3da3894ac0502c9
I20260808 23:40:27.011786 27257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 23:40:27.011946 27257 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Starting leader election (detected failure of leader b98e6c34670f4565a3da3894ac0502c9)
I20260808 23:40:27.012048 27257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:40:27.014829 27257 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } }
I20260808 23:40:27.015518 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:26.996844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:39493 (local address 127.0.0.1:56514)
0808 23:40:27.008549 (+ 11705us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.008567 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:27.008786 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:27.009204 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:27.009210 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:27.009239 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:27.009730 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.009748 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.011302 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.011307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.013384 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.013410 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.014699 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.014705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.014911 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.014946 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:40:27.015054 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 23:40:27.015190 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11608,"thread_start_us":10373,"threads_started":1}
I20260808 23:40:27.015811 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.006522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.18.186.188:39493)
0808 23:40:27.008095 (+ 1573us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:27.008100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:27.008728 (+ 628us) server_negotiation.cc:407] Connection header received
0808 23:40:27.008977 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:27.008981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:27.009036 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:27.009165 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:27.009938 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.011113 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.013547 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.014505 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.014648 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.014699 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.015107 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.015244 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1424,"thread_start_us":1284,"threads_started":1}
I20260808 23:40:27.016117 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.015687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35504)
0808 23:40:27.015823 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.015934 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 23:40:27.016471 27257 leader_election.cc:288] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [CANDIDATE]: Term 2 election: Requested vote from peers b98e6c34670f4565a3da3894ac0502c9 (127.18.186.190:42189), fb4bc7131cf44e9a87eea91b293def91 (127.18.186.189:39769)
I20260808 23:40:27.016434 26667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fb4bc7131cf44e9a87eea91b293def91" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "efd512694f1846a0935af075037a343d" is_pre_election: true
I20260808 23:40:27.016855 26667 raft_consensus.cc:2379] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb4bc7131cf44e9a87eea91b293def91 in current term 2: Already voted for candidate efd512694f1846a0935af075037a343d in this term.
I20260808 23:40:27.017266 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.016758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35516)
0808 23:40:27.017018 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.017134 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260808 23:40:27.017318 26464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efd512694f1846a0935af075037a343d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fb4bc7131cf44e9a87eea91b293def91"
W20260808 23:40:27.017488 26641 leader_election.cc:334] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b98e6c34670f4565a3da3894ac0502c9 (127.18.186.190:42189): Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
I20260808 23:40:27.017638 26464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:40:27.017582 26437 leader_election.cc:302] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb4bc7131cf44e9a87eea91b293def91; no voters: b98e6c34670f4565a3da3894ac0502c9, efd512694f1846a0935af075037a343d
I20260808 23:40:27.019974 26464 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd512694f1846a0935af075037a343d in term 2.
I20260808 23:40:27.020144 27258 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:40:27.020390 26637 leader_election.cc:302] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: efd512694f1846a0935af075037a343d, fb4bc7131cf44e9a87eea91b293def91; no voters: b98e6c34670f4565a3da3894ac0502c9
I20260808 23:40:27.020695 27257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:40:27.021143 27257 raft_consensus.cc:686] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [term 2 LEADER]: Becoming Leader. State: Replica: efd512694f1846a0935af075037a343d, State: Running, Role: LEADER
I20260808 23:40:27.021517 27257 consensus_queue.cc:237] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } }
I20260808 23:40:27.023501 27268 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd512694f1846a0935af075037a343d. Latest consensus state: current_term: 2 leader_uuid: "efd512694f1846a0935af075037a343d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:27.023941 27268 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:27.026029 27270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:27.029858 27270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:27.030723 27270 catalog_manager.cc:1259] Loaded cluster ID: 79856681b2aa48eda54637c35a39f9e0
I20260808 23:40:27.030825 27270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:27.032047 27270 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:27.032842 27270 catalog_manager.cc:5879] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Loaded TSK: 0
I20260808 23:40:27.033716 27270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:40:27.082815 26657 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:40:27.090710 26657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:27.094981 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.094480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35538)
0808 23:40:27.094698 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.094794 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 23:40:27.094969 26464 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [term 2 FOLLOWER]: Refusing update from remote peer efd512694f1846a0935af075037a343d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 23:40:27.095800 27268 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 23:40:27.096127 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.095676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35546)
0808 23:40:27.095876 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.095959 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 23:40:27.096350 26641 consensus_peers.cc:487] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d -> Peer b98e6c34670f4565a3da3894ac0502c9 (127.18.186.190:42189): Couldn't send request to peer b98e6c34670f4565a3da3894ac0502c9. Status: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 23:40:27.098496 27258 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd512694f1846a0935af075037a343d. Latest consensus state: current_term: 2 leader_uuid: "efd512694f1846a0935af075037a343d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:27.098783 27258 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:40:27.100324 27257 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "efd512694f1846a0935af075037a343d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:27.100776 27257 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:27.101235 27258 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "efd512694f1846a0935af075037a343d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b98e6c34670f4565a3da3894ac0502c9" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42189 } } peers { permanent_uuid: "fb4bc7131cf44e9a87eea91b293def91" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 39769 } attrs { promote: false } } peers { permanent_uuid: "efd512694f1846a0935af075037a343d" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 39493 } attrs { promote: false } } }
I20260808 23:40:27.101576 27258 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4bc7131cf44e9a87eea91b293def91 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:40:27.119783 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.110006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:39299 (local address 127.0.0.1:38688)
0808 23:40:27.110169 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.110184 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:27.110281 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:27.111654 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:27.111659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:27.111694 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:27.112124 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.112141 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.114072 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.114081 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.116716 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.116737 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.118857 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.118867 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.119129 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.119189 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 23:40:27.119270 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 23:40:27.119490 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 23:40:27.120810 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.110336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.18.186.130:39299)
0808 23:40:27.111001 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:27.111011 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:27.111035 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:40:27.111119 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:27.111127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:27.111470 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:27.111606 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:27.112374 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.113875 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.116978 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.118582 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.120177 (+ 1595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.120265 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.120315 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.120478 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":297,"threads_started":1}
I20260808 23:40:27.121562 26944 tablet_service.cc:1467] Processing CreateTablet for tablet eda90b8180f041b29cb38744930833aa (DEFAULT_TABLE table=test-table [id=4a316ff4fe5e45e597edf0546035f953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:27.122092 26944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda90b8180f041b29cb38744930833aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:27.126120 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.110415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37883 (local address 127.0.0.1:57078)
0808 23:40:27.111119 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.111145 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:27.111312 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:27.120256 (+ 8944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:27.120261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:27.120289 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:27.120614 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.120625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.122245 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.122250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.123683 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.123697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.125363 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.125368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.125548 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.125582 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:40:27.125646 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 23:40:27.125777 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":601,"threads_started":1}
I20260808 23:40:27.126884 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.110813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57078 (local address 127.18.186.131:37883)
0808 23:40:27.119807 (+ 8994us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:27.119815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:27.119837 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:40:27.119924 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:27.119931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:27.120021 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:27.120789 (+ 768us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:27.121012 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.122069 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.123875 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.125103 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.125868 (+ 765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.125943 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.125996 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.126141 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8858,"thread_start_us":9284,"threads_started":1}
I20260808 23:40:27.127622 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.109560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45533 (local address 127.0.0.1:48448)
0808 23:40:27.109739 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.109758 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:27.109879 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:27.121611 (+ 11732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:27.121616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:27.121640 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:27.121801 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.121811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.123376 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.123381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.125358 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:27.125370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.126836 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.126841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:27.127016 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.127051 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:40:27.127154 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 23:40:27.127291 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 23:40:27.127832 27104 tablet_service.cc:1467] Processing CreateTablet for tablet eda90b8180f041b29cb38744930833aa (DEFAULT_TABLE table=test-table [id=4a316ff4fe5e45e597edf0546035f953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:27.128033 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.110150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48448 (local address 127.18.186.129:45533)
0808 23:40:27.121222 (+ 11072us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:27.121228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:27.121248 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:40:27.121315 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:27.121319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:27.121388 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:27.122010 (+ 622us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:27.122222 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.123222 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.125547 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.126663 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.126846 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.126932 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.127544 (+ 612us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.127704 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10949,"thread_start_us":10867,"threads_started":1}
I20260808 23:40:27.128324 27104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda90b8180f041b29cb38744930833aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:27.128405 27282 tablet_bootstrap.cc:492] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: Bootstrap starting.
I20260808 23:40:27.128769 26777 tablet_service.cc:1467] Processing CreateTablet for tablet eda90b8180f041b29cb38744930833aa (DEFAULT_TABLE table=test-table [id=4a316ff4fe5e45e597edf0546035f953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:27.129202 26777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eda90b8180f041b29cb38744930833aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:27.130223 27282 tablet_bootstrap.cc:654] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:27.133514 27284 tablet_bootstrap.cc:492] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: Bootstrap starting.
I20260808 23:40:27.135069 27282 tablet_bootstrap.cc:492] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: No bootstrap required, opened a new log
I20260808 23:40:27.135216 27282 ts_tablet_manager.cc:1397] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260808 23:40:27.135339 27284 tablet_bootstrap.cc:654] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:27.136047 27282 raft_consensus.cc:348] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.136315 27282 raft_consensus.cc:374] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:27.136435 27282 raft_consensus.cc:729] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69dc5d50059845138c68d58c469763e5, State: Initialized, Role: FOLLOWER
I20260808 23:40:27.136677 27282 consensus_queue.cc:260] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.137328 27286 tablet_bootstrap.cc:492] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: Bootstrap starting.
I20260808 23:40:27.138168 27284 tablet_bootstrap.cc:492] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: No bootstrap required, opened a new log
I20260808 23:40:27.138326 27284 ts_tablet_manager.cc:1397] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 23:40:27.139212 27284 raft_consensus.cc:348] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.139415 27284 raft_consensus.cc:374] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:27.139532 27284 raft_consensus.cc:729] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa6b0c648d64bbe8675e8f4cc891256, State: Initialized, Role: FOLLOWER
I20260808 23:40:27.139806 27284 consensus_queue.cc:260] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.140710 27284 ts_tablet_manager.cc:1428] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 23:40:27.144815 27286 tablet_bootstrap.cc:654] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:27.145696 27175 heartbeater.cc:503] Master 127.18.186.188:39493 was elected leader, sending a full tablet report...
I20260808 23:40:27.148664 27286 tablet_bootstrap.cc:492] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: No bootstrap required, opened a new log
I20260808 23:40:27.148839 27286 ts_tablet_manager.cc:1397] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260808 23:40:27.149502 27286 raft_consensus.cc:348] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.149708 27286 raft_consensus.cc:374] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:27.149807 27286 raft_consensus.cc:729] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e55cc3d8604e7ea8b29d95071a837a, State: Initialized, Role: FOLLOWER
I20260808 23:40:27.150081 27286 consensus_queue.cc:260] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.151201 27282 ts_tablet_manager.cc:1428] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: Time spent starting tablet: real 0.016s user 0.009s sys 0.008s
I20260808 23:40:27.153009 27017 heartbeater.cc:503] Master 127.18.186.188:39493 was elected leader, sending a full tablet report...
I20260808 23:40:27.155251 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.141370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.131:44361)
0808 23:40:27.154752 (+ 13382us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.154954 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13245,"thread_start_us":13103,"threads_started":1}
I20260808 23:40:27.155790 27286 ts_tablet_manager.cc:1428] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260808 23:40:27.156127 26851 heartbeater.cc:503] Master 127.18.186.188:39493 was elected leader, sending a full tablet report...
I20260808 23:40:27.160274 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.157590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.131:49449)
0808 23:40:27.160015 (+ 2425us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.160118 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2312}
I20260808 23:40:27.161196 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.152414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.129:42731)
0808 23:40:27.160050 (+ 7636us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.160259 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7430,"thread_start_us":7245,"threads_started":1}
I20260808 23:40:27.162185 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.151965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.130:42985)
0808 23:40:27.161690 (+ 9725us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.161916 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9580,"thread_start_us":9413,"threads_started":1}
I20260808 23:40:27.162751 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.162411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.129:52751)
0808 23:40:27.162542 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.162626 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 23:40:27.163216 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.162826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.130:58865)
0808 23:40:27.162994 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.163091 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 23:40:27.334921 27288 raft_consensus.cc:482] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:27.335186 27288 raft_consensus.cc:504] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.336063 27288 leader_election.cc:288] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533), 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299)
I20260808 23:40:27.336735 26968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda90b8180f041b29cb38744930833aa" candidate_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69dc5d50059845138c68d58c469763e5" is_pre_election: true
I20260808 23:40:27.336741 26800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda90b8180f041b29cb38744930833aa" candidate_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" is_pre_election: true
I20260808 23:40:27.337033 26968 raft_consensus.cc:2454] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa6b0c648d64bbe8675e8f4cc891256 in term 0.
I20260808 23:40:27.337080 26800 raft_consensus.cc:2454] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa6b0c648d64bbe8675e8f4cc891256 in term 0.
I20260808 23:40:27.337536 27054 leader_election.cc:302] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69dc5d50059845138c68d58c469763e5, dfa6b0c648d64bbe8675e8f4cc891256; no voters:
I20260808 23:40:27.337798 27288 raft_consensus.cc:2781] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:27.337918 27288 raft_consensus.cc:482] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:27.338012 27288 raft_consensus.cc:3037] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:27.340116 27288 raft_consensus.cc:504] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.340684 27288 leader_election.cc:288] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [CANDIDATE]: Term 1 election: Requested vote from peers b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533), 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299)
I20260808 23:40:27.341130 26800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda90b8180f041b29cb38744930833aa" candidate_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e55cc3d8604e7ea8b29d95071a837a"
I20260808 23:40:27.341177 26968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eda90b8180f041b29cb38744930833aa" candidate_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69dc5d50059845138c68d58c469763e5"
I20260808 23:40:27.341327 26800 raft_consensus.cc:3037] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:27.341364 26968 raft_consensus.cc:3037] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:27.343336 26968 raft_consensus.cc:2454] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa6b0c648d64bbe8675e8f4cc891256 in term 1.
I20260808 23:40:27.343401 26800 raft_consensus.cc:2454] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa6b0c648d64bbe8675e8f4cc891256 in term 1.
I20260808 23:40:27.343762 27054 leader_election.cc:302] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69dc5d50059845138c68d58c469763e5, dfa6b0c648d64bbe8675e8f4cc891256; no voters:
I20260808 23:40:27.344066 27288 raft_consensus.cc:2781] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:27.344514 27288 raft_consensus.cc:686] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 LEADER]: Becoming Leader. State: Replica: dfa6b0c648d64bbe8675e8f4cc891256, State: Running, Role: LEADER
I20260808 23:40:27.344895 27288 consensus_queue.cc:237] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } }
I20260808 23:40:27.345639 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.345116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.131:43957)
0808 23:40:27.345328 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.345457 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 23:40:27.348166 26656 catalog_manager.cc:5495] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfa6b0c648d64bbe8675e8f4cc891256 (127.18.186.131). New cstate: current_term: 1 leader_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfa6b0c648d64bbe8675e8f4cc891256" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37883 } health_report { overall_health: HEALTHY } } }
I20260808 23:40:27.415315 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.410925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57088 (local address 127.18.186.131:37883)
0808 23:40:27.411106 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:27.411110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:27.411128 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:40:27.411206 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:27.411210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:27.411275 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:27.411351 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:27.411758 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.412578 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.413825 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.414237 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.414326 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.414680 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:27.414768 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:27.414830 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.414961 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.415061 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:40:27.418367 26968 raft_consensus.cc:1261] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 1 FOLLOWER]: Refusing update from remote peer dfa6b0c648d64bbe8675e8f4cc891256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:27.418440 26800 raft_consensus.cc:1261] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 1 FOLLOWER]: Refusing update from remote peer dfa6b0c648d64bbe8675e8f4cc891256: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:27.419091 27294 consensus_queue.cc:1035] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69dc5d50059845138c68d58c469763e5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 39299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:27.419474 27288 consensus_queue.cc:1035] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e55cc3d8604e7ea8b29d95071a837a" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:27.423123 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.422469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.129:40133)
0808 23:40:27.422705 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.422862 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 23:40:27.423149 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.421775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.130:40095)
0808 23:40:27.422781 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.422888 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":896}
I20260808 23:40:27.431063 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.430571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.129:52927)
0808 23:40:27.430793 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.430900 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 23:40:27.431867 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.431187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.131:59427)
0808 23:40:27.431582 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.431707 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259}
I20260808 23:40:27.438789 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.438335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.131:60061)
0808 23:40:27.438498 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.438608 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 23:40:27.440618 26656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:27.440886 26656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:27.441910 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.441407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.18.186.130:40381)
0808 23:40:27.441644 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.441735 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260808 23:40:27.448746 26656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efd512694f1846a0935af075037a343d: Sending DeleteTablet for 3 replicas of tablet eda90b8180f041b29cb38744930833aa
I20260808 23:40:27.449079 26944 tablet_service.cc:1514] Processing DeleteTablet for tablet eda90b8180f041b29cb38744930833aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:27 UTC) from {username='oryx', principal='oryx/127.18.186.188@KRBTEST.COM'} at 127.0.0.1:38688
I20260808 23:40:27.449587 26777 tablet_service.cc:1514] Processing DeleteTablet for tablet eda90b8180f041b29cb38744930833aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:27 UTC) from {username='oryx', principal='oryx/127.18.186.188@KRBTEST.COM'} at 127.0.0.1:48448
I20260808 23:40:27.450062 27104 tablet_service.cc:1514] Processing DeleteTablet for tablet eda90b8180f041b29cb38744930833aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:27 UTC) from {username='oryx', principal='oryx/127.18.186.188@KRBTEST.COM'} at 127.0.0.1:57078
I20260808 23:40:27.450300 26942 tablet_service.cc:1514] Processing DeleteTablet for tablet eda90b8180f041b29cb38744930833aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:27 UTC) from {username='oryx', principal='oryx/127.18.186.188@KRBTEST.COM'} at 127.0.0.1:38688
I20260808 23:40:27.452673 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 26628
I20260808 23:40:27.456377 27309 tablet_replica.cc:330] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: stopping tablet replica
I20260808 23:40:27.456645 27309 raft_consensus.cc:2229] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:27.456983 27309 raft_consensus.cc:2258] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:27.458315 27309 ts_tablet_manager.cc:1905] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:27.465627 27310 tablet_replica.cc:330] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: stopping tablet replica
I20260808 23:40:27.465756 27307 tablet_replica.cc:330] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: stopping tablet replica
I20260808 23:40:27.465852 27310 raft_consensus.cc:2229] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:40:27.465965 27307 raft_consensus.cc:2229] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:27.466260 27307 raft_consensus.cc:2258] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:27.466296 27310 raft_consensus.cc:2258] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:27.466324 27309 ts_tablet_manager.cc:1918] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:27.466627 27309 log.cc:1196] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/eda90b8180f041b29cb38744930833aa
I20260808 23:40:27.467175 27309 ts_tablet_manager.cc:1939] T eda90b8180f041b29cb38744930833aa P b2e55cc3d8604e7ea8b29d95071a837a: Deleting consensus metadata
I20260808 23:40:27.467476 27310 ts_tablet_manager.cc:1905] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:27.467466 27307 ts_tablet_manager.cc:1905] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:27.468652 26640 catalog_manager.cc:4841] TS b2e55cc3d8604e7ea8b29d95071a837a (127.18.186.129:45533): tablet eda90b8180f041b29cb38744930833aa (table test-table [id=4a316ff4fe5e45e597edf0546035f953]) successfully deleted
I20260808 23:40:27.473969 27307 ts_tablet_manager.cc:1918] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:27.473968 27310 ts_tablet_manager.cc:1918] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:27.474109 27307 log.cc:1196] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/eda90b8180f041b29cb38744930833aa
I20260808 23:40:27.474108 27310 log.cc:1196] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/eda90b8180f041b29cb38744930833aa
I20260808 23:40:27.474212 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.453702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.18.186.129:45533)
0808 23:40:27.453953 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:27.453961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:27.457646 (+ 3685us) server_negotiation.cc:407] Connection header received
0808 23:40:27.457876 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:27.457881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:27.457975 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:27.458050 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:27.460023 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.461121 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.463052 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.463701 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.463808 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.467500 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:27.467521 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:27.467541 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:27.467582 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:27.471247 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:27.472009 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:27.472018 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:27.472024 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:27.472165 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:27.472570 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:27.472575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:27.472580 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:27.472941 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:27.473279 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.473619 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.473843 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:40:27.474579 27307 ts_tablet_manager.cc:1939] T eda90b8180f041b29cb38744930833aa P 69dc5d50059845138c68d58c469763e5: Deleting consensus metadata
I20260808 23:40:27.474579 27310 ts_tablet_manager.cc:1939] T eda90b8180f041b29cb38744930833aa P dfa6b0c648d64bbe8675e8f4cc891256: Deleting consensus metadata
I20260808 23:40:27.474911 26844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:27.475811 26640 catalog_manager.cc:4841] TS dfa6b0c648d64bbe8675e8f4cc891256 (127.18.186.131:37883): tablet eda90b8180f041b29cb38744930833aa (table test-table [id=4a316ff4fe5e45e597edf0546035f953]) successfully deleted
W20260808 23:40:27.476347 26641 catalog_manager.cc:4808] TS 69dc5d50059845138c68d58c469763e5 (127.18.186.130:39299): delete failed for tablet eda90b8180f041b29cb38744930833aa because the tablet was not found. No further retry: Not found: Tablet not found: eda90b8180f041b29cb38744930833aa
I20260808 23:40:27.535926 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.535382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35556)
0808 23:40:27.535622 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.535740 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 23:40:27.537009 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.536619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42189 (local address 127.0.0.1:35566)
0808 23:40:27.536763 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:27.536846 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 23:40:27.576262 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 26628
I20260808 23:40:27.597172 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 26864
I20260808 23:40:27.610451 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.597948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.18.186.130:39299)
0808 23:40:27.598132 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:27.598136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:27.598155 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:40:27.598322 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:27.598326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:27.598417 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:27.598501 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:27.599907 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.600598 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.601969 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.602397 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.602494 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.605159 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:27.605198 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:27.605222 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:27.605269 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:27.608070 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:27.608686 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:27.608693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:27.608695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:27.608811 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:27.609107 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:27.609111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:27.609112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:27.609448 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:27.609661 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.609918 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.610087 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:40:27.611155 27012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:27.714607 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 26864
I20260808 23:40:27.734724 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 27032
I20260808 23:40:27.748378 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:27.735578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.18.186.131:37883)
0808 23:40:27.735770 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:27.735777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:27.735870 (+ 93us) server_negotiation.cc:407] Connection header received
0808 23:40:27.736101 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:27.736106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:27.736204 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:27.736313 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:27.737667 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.738355 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.739613 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:27.740002 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:27.740133 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:27.742588 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:27.742602 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:27.742619 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:27.742657 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:27.745574 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:27.746244 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:27.746252 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:27.746256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:27.746386 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:27.746772 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:27.746778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:27.746782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:27.747118 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:27.747491 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:27.747843 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 23:40:27.748070 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:40:27.749233 27170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:27.871114 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 27032
I20260808 23:40:27.896098 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 26301
I20260808 23:40:27.897260 26486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:28.016063 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 26301
I20260808 23:40:28.039983 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 26415
I20260808 23:40:28.041061 26687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:28.164224 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 26415
2026-08-08T23:40:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10231 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:40:28 dist-test-slave-5k54 krb5kdc[27321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:40:28 dist-test-slave-5k54 krb5kdc[27321](info): set up 2 sockets
Aug 08 23:40:28 dist-test-slave-5k54 krb5kdc[27321](info): commencing operation
krb5kdc: starting...
W20260808 23:40:29.327975 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:40:29 dist-test-slave-5k54 krb5kdc[27321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.190.keytab.
Entry for principal oryx/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.190.keytab.
I20260808 23:40:29.580549 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:34631
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.190.keytab
--principal=oryx/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:34631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:29.925266 27335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:29.925547 27335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:29.925637 27335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:29.932672 27335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:29.932775 27335 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 23:40:29.932816 27335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:29.932875 27335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:29.932925 27335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:29.932981 27335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:29.942893 27335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:34631
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.190.keytab
--principal=oryx/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:34631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:29.944310 27335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:29.945812 27335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:29.957835 27340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:29.957844 27341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:29.958536 27343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:29.958657 27335 server_base.cc:1034] running on GCE node
I20260808 23:40:29.959491 27335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:29.960997 27335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:29.962209 27335 hybrid_clock.cc:648] HybridClock initialized: now 1786232429962143 us; error 52 us; skew 500 ppm
I20260808 23:40:29.962584 27335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:29 dist-test-slave-5k54 krb5kdc[27321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232429, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:29.965874 27335 init.cc:377] Logged in from keytab as oryx/127.18.186.190@KRBTEST.COM (short username oryx)
I20260808 23:40:29.967338 27335 webserver.cc:460] Webserver started at http://127.18.186.190:41829/ using document root <none> and password file <none>
I20260808 23:40:29.968053 27335 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:29.968140 27335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:29.968400 27335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:29.971226 27335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "a772a0fbd3624e51a85ec20e46e2cabf"
format_stamp: "Formatted at 2026-08-08 23:40:29 on dist-test-slave-5k54"
server_key: "796dcd85b646ceb47e0c2326fce52e1c"
server_key_iv: "63623ba033c9b5728ba912c38c88aead"
server_key_version: "encryptionkey@0"
I20260808 23:40:29.972046 27335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "a772a0fbd3624e51a85ec20e46e2cabf"
format_stamp: "Formatted at 2026-08-08 23:40:29 on dist-test-slave-5k54"
server_key: "796dcd85b646ceb47e0c2326fce52e1c"
server_key_iv: "63623ba033c9b5728ba912c38c88aead"
server_key_version: "encryptionkey@0"
I20260808 23:40:29.976959 27335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:40:29.980285 27350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:29.981578 27335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:40:29.981755 27335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "a772a0fbd3624e51a85ec20e46e2cabf"
format_stamp: "Formatted at 2026-08-08 23:40:29 on dist-test-slave-5k54"
server_key: "796dcd85b646ceb47e0c2326fce52e1c"
server_key_iv: "63623ba033c9b5728ba912c38c88aead"
server_key_version: "encryptionkey@0"
I20260808 23:40:29.981959 27335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:29.997586 27335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:30.001879 27335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:30.002179 27335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:30.024830 27335 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:34631
I20260808 23:40:30.024852 27417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:34631 every 8 connection(s)
I20260808 23:40:30.026206 27335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:30.029970 27418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:30.036459 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 27335
I20260808 23:40:30.036731 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:30.037609 19178 external_mini_cluster.cc:1428] Setting key 5347e7af9c6ce49e5426090cd6cf0436
I20260808 23:40:30.041347 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: Bootstrap starting.
I20260808 23:40:30.044934 27418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:30.046044 27418 log.cc:824] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:30.048785 27418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: No bootstrap required, opened a new log
Aug 08 23:40:30 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:30.056277 27418 raft_consensus.cc:348] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a772a0fbd3624e51a85ec20e46e2cabf" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34631 } }
I20260808 23:40:30.056648 27418 raft_consensus.cc:374] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:30.056752 27418 raft_consensus.cc:729] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a772a0fbd3624e51a85ec20e46e2cabf, State: Initialized, Role: FOLLOWER
I20260808 23:40:30.057438 27418 consensus_queue.cc:260] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a772a0fbd3624e51a85ec20e46e2cabf" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34631 } }
I20260808 23:40:30.057686 27418 raft_consensus.cc:388] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:30.057854 27418 raft_consensus.cc:482] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:30.058065 27418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:30.060645 27418 raft_consensus.cc:504] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a772a0fbd3624e51a85ec20e46e2cabf" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34631 } }
I20260808 23:40:30.060808 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:30.039784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.18.186.190:34631)
0808 23:40:30.040613 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:30.040626 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:30.040669 (+ 43us) server_negotiation.cc:407] Connection header received
0808 23:40:30.042470 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:30.042491 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:30.042879 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:30.043202 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:30.045116 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.046560 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.047909 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.048587 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.048783 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:30.052859 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:30.052867 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:30.052881 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:30.052922 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:30.056224 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:30.057052 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:30.057058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:30.057064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:30.057237 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:30.057655 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:30.057660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:30.057662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:30.058363 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:30.058690 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:30.059159 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 23:40:30.059617 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":380,"threads_started":1}
I20260808 23:40:30.061265 27418 leader_election.cc:302] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a772a0fbd3624e51a85ec20e46e2cabf; no voters:
I20260808 23:40:30.061930 27418 leader_election.cc:288] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:30.062065 27424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:30.062762 27424 raft_consensus.cc:686] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [term 1 LEADER]: Becoming Leader. State: Replica: a772a0fbd3624e51a85ec20e46e2cabf, State: Running, Role: LEADER
I20260808 23:40:30.063329 27424 consensus_queue.cc:237] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a772a0fbd3624e51a85ec20e46e2cabf" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34631 } }
I20260808 23:40:30.065239 27418 sys_catalog.cc:564] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:30.066449 27426 sys_catalog.cc:455] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a772a0fbd3624e51a85ec20e46e2cabf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a772a0fbd3624e51a85ec20e46e2cabf" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34631 } } }
I20260808 23:40:30.066879 27426 sys_catalog.cc:458] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:30.070169 27425 sys_catalog.cc:455] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a772a0fbd3624e51a85ec20e46e2cabf. Latest consensus state: current_term: 1 leader_uuid: "a772a0fbd3624e51a85ec20e46e2cabf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a772a0fbd3624e51a85ec20e46e2cabf" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34631 } } }
I20260808 23:40:30.070465 27425 sys_catalog.cc:458] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:30.072371 27430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:30.076838 27430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:30.086555 27430 catalog_manager.cc:1347] Generated new cluster ID: cebaf8ca7a6c4af9b7e265ebb48be17c
I20260808 23:40:30.086661 27430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:30.103334 27430 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:30.104655 27430 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:30.114827 27430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: Generated new TSK 0
I20260808 23:40:30.116080 27430 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab.
Entry for principal oryx/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab.
I20260808 23:40:30.181216 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34631
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab
--principal=oryx/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:30.514005 27446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:30.514290 27446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:30.514467 27446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:30.521654 27446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:30.521762 27446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:30.521955 27446 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:40:30.522085 27446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:30.531839 27446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab
--principal=oryx/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:34631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:30.533134 27446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:30.534591 27446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:30.547652 27451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:30.547554 27455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:30.547580 27452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:30.548693 27446 server_base.cc:1034] running on GCE node
I20260808 23:40:30.549314 27446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:30.550191 27446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:30.551484 27446 hybrid_clock.cc:648] HybridClock initialized: now 1786232430551371 us; error 44 us; skew 500 ppm
I20260808 23:40:30.551781 27446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:30 dist-test-slave-5k54 krb5kdc[27321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232430, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:30.554972 27446 init.cc:377] Logged in from keytab as oryx/127.18.186.129@KRBTEST.COM (short username oryx)
I20260808 23:40:30.556432 27446 webserver.cc:460] Webserver started at http://127.18.186.129:38017/ using document root <none> and password file <none>
I20260808 23:40:30.557121 27446 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:30.557252 27446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:30.557576 27446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:30.560448 27446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "277f0ee0121b49afb3213a37ca474631"
format_stamp: "Formatted at 2026-08-08 23:40:30 on dist-test-slave-5k54"
server_key: "742c121e529c9966d903c1a9a39d75f5"
server_key_iv: "41d58f6711efaa89f6211b0e3939c79b"
server_key_version: "encryptionkey@0"
I20260808 23:40:30.561339 27446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "277f0ee0121b49afb3213a37ca474631"
format_stamp: "Formatted at 2026-08-08 23:40:30 on dist-test-slave-5k54"
server_key: "742c121e529c9966d903c1a9a39d75f5"
server_key_iv: "41d58f6711efaa89f6211b0e3939c79b"
server_key_version: "encryptionkey@0"
I20260808 23:40:30.566941 27446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 23:40:30.570446 27462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:30.571882 27446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:30.572077 27446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "277f0ee0121b49afb3213a37ca474631"
format_stamp: "Formatted at 2026-08-08 23:40:30 on dist-test-slave-5k54"
server_key: "742c121e529c9966d903c1a9a39d75f5"
server_key_iv: "41d58f6711efaa89f6211b0e3939c79b"
server_key_version: "encryptionkey@0"
I20260808 23:40:30.572289 27446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:30.607627 27446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:30.611649 27446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:30.611925 27446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:30.614748 27446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:30.614856 27446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:30.615257 27446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:30.615355 27446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:40:30 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232430, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:30.638357 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:30.615246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:53561 (local address 127.18.186.190:34631)
0808 23:40:30.615826 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:30.615836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:30.616998 (+ 1162us) server_negotiation.cc:407] Connection header received
0808 23:40:30.619093 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:30.619101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:30.619196 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:30.619344 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:30.621376 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.622328 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.625072 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.625707 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.625873 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:30.631405 (+ 5532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:30.631413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:30.631444 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:30.631491 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:30.634857 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:30.635868 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:30.635877 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:30.635882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:30.636071 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:30.636595 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:30.636602 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:30.636610 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:30.636888 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:30.637058 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:30.637760 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 23:40:30.637979 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":344,"threads_started":1}
I20260808 23:40:30.639395 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:30.615987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34631 (local address 127.18.186.129:53561)
0808 23:40:30.616840 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:30.616884 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:30.618826 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:30.619621 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:30.619629 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:30.620361 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:30.621199 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:30.621221 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.622510 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.622514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:30.624679 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:30.624694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.626092 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.626125 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:30.626476 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:30.627357 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:30.627400 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:30.631156 (+ 3756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:30.635004 (+ 3848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:30.635016 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:30.635039 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:30.635699 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:30.636214 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:30.636220 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:30.636226 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:30.636460 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:30.637021 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:30.637031 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:30.637399 (+ 368us) client_negotiation.cc:769] Sending connection context
0808 23:40:30.637665 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 23:40:30.638189 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":405,"threads_started":1}
I20260808 23:40:30.665242 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:30.647113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34631 (local address 127.18.186.129:46295)
0808 23:40:30.647857 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:30.647878 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:30.648038 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:30.648449 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:30.648457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:30.649333 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:30.649919 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:30.649939 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.651303 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.651309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:30.653047 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:30.653058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.653835 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.653841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:30.654028 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:30.654758 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:30.654776 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:30.656112 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:30.662207 (+ 6095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:30.662213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:30.662217 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:30.662663 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:30.663224 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:30.663230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:30.663236 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:30.663408 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:30.664268 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:30.664274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:30.664473 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 23:40:30.664591 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 23:40:30.664848 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":459,"threads_started":1}
I20260808 23:40:30.665750 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:30.646850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:46295 (local address 127.18.186.190:34631)
0808 23:40:30.647045 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:30.647051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:30.647971 (+ 920us) server_negotiation.cc:407] Connection header received
0808 23:40:30.648175 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:30.648183 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:30.648276 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:30.648397 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:30.650185 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.651162 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.653203 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:30.653648 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:30.653793 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:30.656366 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:30.656376 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:30.656385 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:30.656441 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:30.661956 (+ 5515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:30.662831 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:30.662841 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:30.662848 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:30.663051 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:30.663620 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:30.663625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:30.663629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:30.664109 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:30.664313 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:30.664850 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 23:40:30.665207 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:40:30.672561 27446 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:35277
I20260808 23:40:30.672590 27595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:35277 every 8 connection(s)
I20260808 23:40:30.673861 27446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:30.676148 27596 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34631
I20260808 23:40:30.676412 27596 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:30.676460 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 27446
I20260808 23:40:30.676688 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:40:30.677106 27596 heartbeater.cc:511] Master 127.18.186.190:34631 requested a full tablet report, sending...
I20260808 23:40:30.677470 19178 external_mini_cluster.cc:1428] Setting key 5e06383478b6b34cf329eb8389b75fdf
I20260808 23:40:30.679405 27368 ts_manager.cc:194] Registered new tserver with Master: 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277)
I20260808 23:40:30.680940 27368 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:53561
WARNING: no policy specified for oryx/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab.
Entry for principal oryx/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab.
I20260808 23:40:30.731536 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34631
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab
--principal=oryx/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:31.064246 27600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:31.064513 27600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:31.064610 27600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:31.071883 27600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:31.071990 27600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:31.072074 27600 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:40:31.072157 27600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:31.081806 27600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab
--principal=oryx/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:34631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:31.083436 27600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:31.084677 27600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:31.097414 27608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:31.097430 27609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:31.097433 27611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:31.098292 27600 server_base.cc:1034] running on GCE node
I20260808 23:40:31.098855 27600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:31.099740 27600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:31.100987 27600 hybrid_clock.cc:648] HybridClock initialized: now 1786232431100927 us; error 40 us; skew 500 ppm
I20260808 23:40:31.101282 27600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:31 dist-test-slave-5k54 krb5kdc[27321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232431, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:31.104403 27600 init.cc:377] Logged in from keytab as oryx/127.18.186.130@KRBTEST.COM (short username oryx)
I20260808 23:40:31.105906 27600 webserver.cc:460] Webserver started at http://127.18.186.130:42497/ using document root <none> and password file <none>
I20260808 23:40:31.106611 27600 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:31.106740 27600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:31.107067 27600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:31.110126 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "3a42ff2112fb48d493827466edf7b510"
format_stamp: "Formatted at 2026-08-08 23:40:31 on dist-test-slave-5k54"
server_key: "c12a0d35d8edf2bc35d5117fa3c10e05"
server_key_iv: "51b79de8bb6915e39d10eb608df8befa"
server_key_version: "encryptionkey@0"
I20260808 23:40:31.110956 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "3a42ff2112fb48d493827466edf7b510"
format_stamp: "Formatted at 2026-08-08 23:40:31 on dist-test-slave-5k54"
server_key: "c12a0d35d8edf2bc35d5117fa3c10e05"
server_key_iv: "51b79de8bb6915e39d10eb608df8befa"
server_key_version: "encryptionkey@0"
I20260808 23:40:31.116317 27600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 23:40:31.119751 27619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:31.121124 27600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:40:31.121308 27600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "3a42ff2112fb48d493827466edf7b510"
format_stamp: "Formatted at 2026-08-08 23:40:31 on dist-test-slave-5k54"
server_key: "c12a0d35d8edf2bc35d5117fa3c10e05"
server_key_iv: "51b79de8bb6915e39d10eb608df8befa"
server_key_version: "encryptionkey@0"
I20260808 23:40:31.121560 27600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:31.138181 27600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:31.142216 27600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:31.142501 27600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:31.145377 27600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:31.145500 27600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:31.145659 27600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:31.145728 27600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:40:31 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232431, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:31.169450 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.146078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:47125 (local address 127.18.186.190:34631)
0808 23:40:31.146274 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.146281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.147455 (+ 1174us) server_negotiation.cc:407] Connection header received
0808 23:40:31.149570 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.149578 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.149686 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.149811 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:31.152106 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.153391 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.156155 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.156619 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.156757 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.161878 (+ 5121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:31.161886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:31.161894 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:31.161953 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:31.165923 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.166714 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.166719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.166723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.166853 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.167341 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.167346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.167349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.167714 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:31.167921 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.168992 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.169161 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:40:31.171388 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.146473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34631 (local address 127.18.186.130:47125)
0808 23:40:31.147256 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.147296 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.149166 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.150114 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.150131 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.150960 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:31.151833 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.151852 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.153586 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.153591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.156008 (+ 2417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.156020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.156873 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.156885 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.157193 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.158119 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:31.158148 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:31.161574 (+ 3426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:31.166070 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:31.166079 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:31.166102 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:31.166584 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:31.167020 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:31.167024 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:31.167026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:31.167215 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:31.167874 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:31.167886 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:31.168493 (+ 607us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.168892 (+ 399us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.169576 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":326,"threads_started":1}
I20260808 23:40:31.197856 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.182151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:54149 (local address 127.18.186.190:34631)
0808 23:40:31.182319 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.182323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.182956 (+ 633us) server_negotiation.cc:407] Connection header received
0808 23:40:31.183153 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.183157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.183215 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.183316 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:31.184950 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.185935 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.187348 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.187925 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.188067 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.190633 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:31.190638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:31.190641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:31.190671 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:31.194316 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.195513 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.195518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.195521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.195697 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.196169 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.196177 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.196181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.196688 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:31.196840 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.197275 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.197488 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:40:31.197979 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.182251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34631 (local address 127.18.186.130:54149)
0808 23:40:31.182847 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.182865 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.183028 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.183340 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.183345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.183999 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:31.184728 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.184748 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.186085 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.186089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.187211 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.187223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.188163 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.188174 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.188416 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.189194 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:31.189225 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:31.190441 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:31.194491 (+ 4050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:31.194499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:31.194506 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:31.195276 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:31.195851 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:31.195858 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:31.195864 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:31.196048 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:31.196825 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:31.196830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:31.197034 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.197182 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.197515 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":344,"threads_started":1}
I20260808 23:40:31.202527 27600 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:35297
I20260808 23:40:31.202548 27745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:35297 every 8 connection(s)
I20260808 23:40:31.203758 27600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:31.205130 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 27600
I20260808 23:40:31.205312 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:40:31.205766 19178 external_mini_cluster.cc:1428] Setting key eb00271ff2c7d8961fff3b5589eb242f
I20260808 23:40:31.205698 27746 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34631
I20260808 23:40:31.207011 27746 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:31.207859 27746 heartbeater.cc:511] Master 127.18.186.190:34631 requested a full tablet report, sending...
I20260808 23:40:31.209048 27368 ts_manager.cc:194] Registered new tserver with Master: 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297)
I20260808 23:40:31.209815 27368 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:47125
WARNING: no policy specified for oryx/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab.
Entry for principal oryx/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab.
I20260808 23:40:31.259248 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34631
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab
--principal=oryx/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:34181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:31.592305 27750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:31.592584 27750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:31.592680 27750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:31.599860 27750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:31.599969 27750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:31.600054 27750 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:40:31.600134 27750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:31.609887 27750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:34181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab
--principal=oryx/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.186.190:34631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:31.611063 27750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:31.612567 27750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:31.625428 27756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:31.625481 27757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:31.625666 27759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:31.626567 27750 server_base.cc:1034] running on GCE node
I20260808 23:40:31.627218 27750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:31.628201 27750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:31.629453 27750 hybrid_clock.cc:648] HybridClock initialized: now 1786232431629386 us; error 43 us; skew 500 ppm
I20260808 23:40:31.629753 27750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:31 dist-test-slave-5k54 krb5kdc[27321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232431, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:31.637529 27750 init.cc:377] Logged in from keytab as oryx/127.18.186.131@KRBTEST.COM (short username oryx)
I20260808 23:40:31.639097 27750 webserver.cc:460] Webserver started at http://127.18.186.131:38909/ using document root <none> and password file <none>
I20260808 23:40:31.639880 27750 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:31.639972 27750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:31.640245 27750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:31.643750 27750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "caee5f5f104b4f85a40c49876141376b"
format_stamp: "Formatted at 2026-08-08 23:40:31 on dist-test-slave-5k54"
server_key: "7c12960f69f5ceee51683adca92c023e"
server_key_iv: "4347e3432b3f30c4d539898d862649b5"
server_key_version: "encryptionkey@0"
I20260808 23:40:31.644539 27750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "caee5f5f104b4f85a40c49876141376b"
format_stamp: "Formatted at 2026-08-08 23:40:31 on dist-test-slave-5k54"
server_key: "7c12960f69f5ceee51683adca92c023e"
server_key_iv: "4347e3432b3f30c4d539898d862649b5"
server_key_version: "encryptionkey@0"
I20260808 23:40:31.649824 27750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:40:31.653471 27767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:31.654784 27750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:40:31.654929 27750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "caee5f5f104b4f85a40c49876141376b"
format_stamp: "Formatted at 2026-08-08 23:40:31 on dist-test-slave-5k54"
server_key: "7c12960f69f5ceee51683adca92c023e"
server_key_iv: "4347e3432b3f30c4d539898d862649b5"
server_key_version: "encryptionkey@0"
I20260808 23:40:31.655105 27750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:31.670739 27750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:31.675509 27750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:31.675791 27750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:31.679059 27750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:31.679147 27750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:31.679303 27750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:31.679386 27750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:31.683705 27596 heartbeater.cc:503] Master 127.18.186.190:34631 was elected leader, sending a full tablet report...
Aug 08 23:40:31 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232431, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:31.704463 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.679763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:52895 (local address 127.18.186.190:34631)
0808 23:40:31.679963 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.679968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.681802 (+ 1834us) server_negotiation.cc:407] Connection header received
0808 23:40:31.684132 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.684138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.684253 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.684345 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:31.687204 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.688559 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.690198 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.691094 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.691249 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.696101 (+ 4852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:31.696110 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:31.696116 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:31.696169 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:31.700213 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.701267 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.701271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.701273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.701437 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.702176 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.702184 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.702189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.702542 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:31.702731 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.703679 (+ 948us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.703963 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:40:31.705705 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.680634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34631 (local address 127.18.186.131:52895)
0808 23:40:31.681576 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.681644 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.683742 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.685013 (+ 1271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.685023 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.685917 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:31.686972 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.687017 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.688739 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.688745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.690050 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.690063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.691341 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.691346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.691600 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.692398 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:31.692430 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:31.695803 (+ 3373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:31.700384 (+ 4581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:31.700394 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:31.700413 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:31.701073 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:31.701581 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:31.701586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:31.701589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:31.701962 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:31.702686 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:31.702694 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:31.703060 (+ 366us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.703389 (+ 329us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.704059 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":381,"threads_started":1}
I20260808 23:40:31.732347 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.713601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34631 (local address 127.18.186.131:42391)
0808 23:40:31.714675 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.714703 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.714926 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.715321 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.715326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.716007 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:31.716290 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.716299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.717899 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.717908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.720292 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.720311 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.721574 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.721583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.721725 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.722412 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:31.722429 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:31.723676 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:31.728619 (+ 4943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:31.728630 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:31.728637 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:31.729488 (+ 851us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:31.730002 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:31.730014 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:31.730020 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:31.730212 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:31.730727 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:31.730732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:31.730979 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.731161 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.731639 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":615,"threads_started":1}
I20260808 23:40:31.733057 27750 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:42865
I20260808 23:40:31.733075 27900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:42865 every 8 connection(s)
I20260808 23:40:31.733392 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.713545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:42391 (local address 127.18.186.190:34631)
0808 23:40:31.713733 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.713741 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.714796 (+ 1055us) server_negotiation.cc:407] Connection header received
0808 23:40:31.715095 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.715101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.715187 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.715292 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:31.716441 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.717697 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.720519 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.721302 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.721447 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.723940 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:31.723946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:31.723952 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:31.723993 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:31.728434 (+ 4441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.729650 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.729657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.729662 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.729846 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.730335 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.730339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.730341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.730605 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:31.730716 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.732743 (+ 2027us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.733032 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:40:31.734586 27750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:31.734879 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 27750
I20260808 23:40:31.735067 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:40:31.735679 19178 external_mini_cluster.cc:1428] Setting key 5638bc2543dfe4c47b4210f683062814
I20260808 23:40:31.739055 27901 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34631
I20260808 23:40:31.739293 27901 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:31.739997 27901 heartbeater.cc:511] Master 127.18.186.190:34631 requested a full tablet report, sending...
I20260808 23:40:31.741276 27368 ts_manager.cc:194] Registered new tserver with Master: caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865)
I20260808 23:40:31.742113 27368 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:52895
I20260808 23:40:31.751387 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:40:31 dist-test-slave-5k54 krb5kdc[27321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232431, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:40:31 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:31 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:40:31.807044 27470 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:31.796829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51580 (local address 127.18.186.190:34631)
0808 23:40:31.797172 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.797180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.797301 (+ 121us) server_negotiation.cc:407] Connection header received
0808 23:40:31.797461 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.797466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.797537 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.797627 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:31.798913 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.799733 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.800896 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.801312 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.801412 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.806784 (+ 5372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51580: BlockingRecv error: recv got EOF from 127.0.0.1:51580 (error 108)
Metrics: {"server-negotiator.queue_time_us":187}
Aug 08 23:40:31 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232431, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:31.824259 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.810921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.18.186.190:34631)
0808 23:40:31.811087 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.811092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.811526 (+ 434us) server_negotiation.cc:407] Connection header received
0808 23:40:31.811691 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.811695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.811775 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.811861 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:31.813008 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.813739 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.814934 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.815340 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.815508 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.819181 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:31.819185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:31.819188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:31.819221 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:31.821879 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.822439 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.822443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.822445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.822557 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:31.823039 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:31.823047 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:31.823049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:31.823319 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:31.823627 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.823832 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.824035 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:40:31.828310 27368 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:40:31.835191 27368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:31.861182 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.846703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:35277 (local address 127.0.0.1:51058)
0808 23:40:31.847914 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.847965 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.849191 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.850285 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.850297 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.850337 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:31.850872 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.850889 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.853453 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.853462 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.856645 (+ 3183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.856686 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.860013 (+ 3327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.860020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.860282 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.860361 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.860515 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.860713 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":4,"thread_start_us":340,"threads_started":1}
I20260808 23:40:31.861426 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.846982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51058 (local address 127.18.186.129:35277)
0808 23:40:31.847937 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.847948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.849135 (+ 1187us) server_negotiation.cc:407] Connection header received
0808 23:40:31.849397 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.849408 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.849730 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.849961 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:31.851122 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.853185 (+ 2063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.856923 (+ 3738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.859389 (+ 2466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.859593 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.859794 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.860933 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.861106 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":626,"threads_started":1}
I20260808 23:40:31.866065 27517 tablet_service.cc:1467] Processing CreateTablet for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 (DEFAULT_TABLE table=test-table [id=8c3c8c87d0dd4907a53da6c745cef9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:31.867172 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.847344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:42865 (local address 127.0.0.1:59266)
0808 23:40:31.848666 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.848690 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.848883 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.860070 (+ 11187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.860074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.860106 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:31.860623 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.860636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.862927 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.862932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.864706 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.864717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.866288 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.866294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.866462 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.866493 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.866587 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.866709 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1161,"mutex_wait_us":2,"spinlock_wait_cycles":18048,"thread_start_us":1029,"threads_started":1}
I20260808 23:40:31.867992 27517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:31.868028 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.847875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.18.186.131:42865)
0808 23:40:31.859288 (+ 11413us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.859300 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.859344 (+ 44us) server_negotiation.cc:407] Connection header received
0808 23:40:31.859500 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.859511 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.859853 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.860837 (+ 984us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:31.861170 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.862748 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.864893 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.866081 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.866266 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.866434 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.867549 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.867714 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11172,"thread_start_us":11609,"threads_started":1}
I20260808 23:40:31.868441 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.847856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:35297 (local address 127.0.0.1:34692)
0808 23:40:31.860123 (+ 12267us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.860141 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.860283 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.861300 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.861306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.861339 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:31.861785 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.861800 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.863199 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.863204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.865497 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.865511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.867585 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.867591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.867761 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.867783 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.867888 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.868010 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12079,"spinlock_wait_cycles":1920,"thread_start_us":11926,"threads_started":1}
I20260808 23:40:31.868518 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.848322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.18.186.130:35297)
0808 23:40:31.859216 (+ 10894us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.859230 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.860432 (+ 1202us) server_negotiation.cc:407] Connection header received
0808 23:40:31.860545 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.860552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.860881 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.861076 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:31.861981 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.863019 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.865656 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.867289 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.867481 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.867655 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.867971 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.868224 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10753,"thread_start_us":10622,"threads_started":1}
I20260808 23:40:31.871805 27822 tablet_service.cc:1467] Processing CreateTablet for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 (DEFAULT_TABLE table=test-table [id=8c3c8c87d0dd4907a53da6c745cef9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:31.872293 27671 tablet_service.cc:1467] Processing CreateTablet for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 (DEFAULT_TABLE table=test-table [id=8c3c8c87d0dd4907a53da6c745cef9da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:31.873327 27822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:31.873588 27671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:31.880782 27925 tablet_bootstrap.cc:492] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Bootstrap starting.
I20260808 23:40:31.884505 27925 tablet_bootstrap.cc:654] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:31.885717 27925 log.cc:824] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:31.888010 27928 tablet_bootstrap.cc:492] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Bootstrap starting.
I20260808 23:40:31.888792 27927 tablet_bootstrap.cc:492] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Bootstrap starting.
I20260808 23:40:31.889300 27925 tablet_bootstrap.cc:492] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: No bootstrap required, opened a new log
I20260808 23:40:31.889590 27925 ts_tablet_manager.cc:1397] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 23:40:31.892633 27927 tablet_bootstrap.cc:654] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:31.893913 27927 log.cc:824] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:31.894191 27928 tablet_bootstrap.cc:654] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:31.895393 27928 log.cc:824] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:31.896983 27927 tablet_bootstrap.cc:492] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: No bootstrap required, opened a new log
I20260808 23:40:31.897251 27927 ts_tablet_manager.cc:1397] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260808 23:40:31.898201 27928 tablet_bootstrap.cc:492] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: No bootstrap required, opened a new log
I20260808 23:40:31.898478 27928 ts_tablet_manager.cc:1397] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.003s
I20260808 23:40:31.900995 27925 raft_consensus.cc:348] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.901539 27925 raft_consensus.cc:374] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:31.901674 27925 raft_consensus.cc:729] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277f0ee0121b49afb3213a37ca474631, State: Initialized, Role: FOLLOWER
I20260808 23:40:31.902345 27925 consensus_queue.cc:260] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.905066 27925 ts_tablet_manager.cc:1428] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260808 23:40:31.911479 27928 raft_consensus.cc:348] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.911943 27928 raft_consensus.cc:374] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:31.912051 27928 raft_consensus.cc:729] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caee5f5f104b4f85a40c49876141376b, State: Initialized, Role: FOLLOWER
I20260808 23:40:31.912118 27927 raft_consensus.cc:348] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.912587 27927 raft_consensus.cc:374] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:31.912703 27927 raft_consensus.cc:729] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a42ff2112fb48d493827466edf7b510, State: Initialized, Role: FOLLOWER
I20260808 23:40:31.912613 27928 consensus_queue.cc:260] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.913287 27927 consensus_queue.cc:260] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.914256 27901 heartbeater.cc:503] Master 127.18.186.190:34631 was elected leader, sending a full tablet report...
I20260808 23:40:31.914604 27746 heartbeater.cc:503] Master 127.18.186.190:34631 was elected leader, sending a full tablet report...
I20260808 23:40:31.914868 27928 ts_tablet_manager.cc:1428] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Time spent starting tablet: real 0.016s user 0.009s sys 0.009s
I20260808 23:40:31.915158 27927 ts_tablet_manager.cc:1428] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
W20260808 23:40:31.926471 27597 tablet.cc:2369] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:40:31.946573 27933 raft_consensus.cc:482] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:31.946848 27933 raft_consensus.cc:504] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.948570 27933 leader_election.cc:288] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865)
I20260808 23:40:31.955789 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.949050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:44309 (local address 127.18.186.131:42865)
0808 23:40:31.949286 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.949294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.950410 (+ 1116us) server_negotiation.cc:407] Connection header received
0808 23:40:31.950556 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.950562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.950650 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.950755 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:31.951393 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.952289 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.954062 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.954792 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.954992 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.955067 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.955373 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.955521 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 23:40:31.955840 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.948912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:41495 (local address 127.18.186.129:35277)
0808 23:40:31.949139 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.949145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.950011 (+ 866us) server_negotiation.cc:407] Connection header received
0808 23:40:31.950262 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.950268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.950364 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.950475 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:31.951132 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.952031 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.953921 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.954756 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.954903 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.954991 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.955403 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.955566 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 23:40:31.956193 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.949515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:42865 (local address 127.18.186.130:44309)
0808 23:40:31.950294 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.950311 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.950444 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.950777 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.950781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.950804 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:31.951208 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.951222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.952436 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.952440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.953916 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.953930 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.955055 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.955060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.955211 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.955235 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.955309 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.955480 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":21,"thread_start_us":380,"threads_started":1}
W20260808 23:40:31.955794 27747 tablet.cc:2369] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:40:31.956614 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.949029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:35277 (local address 127.18.186.130:41495)
0808 23:40:31.949855 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:31.949879 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:31.950077 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:31.950503 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:31.950507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:31.950536 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:31.950976 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.950989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.952170 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.952176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.953760 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:31.953771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.955025 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.955032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:31.955227 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.955250 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:40:31.955361 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 23:40:31.955513 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":19,"thread_start_us":445,"threads_started":1}
I20260808 23:40:31.957461 27851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bea9f3cd0244c7b5bae6c456a3eaf9" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caee5f5f104b4f85a40c49876141376b" is_pre_election: true
I20260808 23:40:31.957584 27544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bea9f3cd0244c7b5bae6c456a3eaf9" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631" is_pre_election: true
I20260808 23:40:31.958076 27851 raft_consensus.cc:2454] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 0.
I20260808 23:40:31.958190 27544 raft_consensus.cc:2454] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 0.
I20260808 23:40:31.958794 27622 leader_election.cc:302] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a42ff2112fb48d493827466edf7b510, caee5f5f104b4f85a40c49876141376b; no voters:
I20260808 23:40:31.959195 27933 raft_consensus.cc:2781] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:31.959342 27933 raft_consensus.cc:482] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:31.959474 27933 raft_consensus.cc:3037] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:31.961891 27933 raft_consensus.cc:504] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.962543 27933 leader_election.cc:288] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 election: Requested vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865)
I20260808 23:40:31.962972 27544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bea9f3cd0244c7b5bae6c456a3eaf9" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631"
I20260808 23:40:31.963071 27851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bea9f3cd0244c7b5bae6c456a3eaf9" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caee5f5f104b4f85a40c49876141376b"
I20260808 23:40:31.963199 27544 raft_consensus.cc:3037] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:31.963250 27851 raft_consensus.cc:3037] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:31.966578 27544 raft_consensus.cc:2454] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 1.
I20260808 23:40:31.966583 27851 raft_consensus.cc:2454] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 1.
I20260808 23:40:31.967038 27622 leader_election.cc:302] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a42ff2112fb48d493827466edf7b510, caee5f5f104b4f85a40c49876141376b; no voters:
I20260808 23:40:31.967355 27933 raft_consensus.cc:2781] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:31.967831 27933 raft_consensus.cc:686] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 1 LEADER]: Becoming Leader. State: Replica: 3a42ff2112fb48d493827466edf7b510, State: Running, Role: LEADER
I20260808 23:40:31.968307 27933 consensus_queue.cc:237] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:31.973246 27368 catalog_manager.cc:5495] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a42ff2112fb48d493827466edf7b510 (127.18.186.130). New cstate: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } health_report { overall_health: UNKNOWN } } }
W20260808 23:40:31.989943 27902 tablet.cc:2369] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:40:31.993844 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:31.988700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.18.186.130:35297)
0808 23:40:31.988893 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:31.988898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:31.989016 (+ 118us) server_negotiation.cc:407] Connection header received
0808 23:40:31.989186 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:31.989189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:31.989268 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:31.989389 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:31.989713 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.990493 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.991885 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:31.992320 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:31.992434 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:31.992799 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:31.993363 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:31.993468 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:31.993577 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 23:40:31.993641 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:40:32.001868 27544 raft_consensus.cc:1261] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Refusing update from remote peer 3a42ff2112fb48d493827466edf7b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:32.001866 27851 raft_consensus.cc:1261] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Refusing update from remote peer 3a42ff2112fb48d493827466edf7b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:32.002979 27936 consensus_queue.cc:1035] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:32.003399 27933 consensus_queue.cc:1035] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:32.017560 27941 mvcc.cc:205] Tried to move back new op lower bound from 7316408041468833792 to 7316408041346338816. Current Snapshot: MvccSnapshot[applied={T|T < 7316408041468833792 or (T in {7316408041468833792})}]
I20260808 23:40:32.029202 27368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:32.029458 27368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:32.033486 27368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: Sending DeleteTablet for 3 replicas of tablet b2bea9f3cd0244c7b5bae6c456a3eaf9
I20260808 23:40:32.034829 27822 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:32 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:59266
I20260808 23:40:32.034991 27517 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:32 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:51058
I20260808 23:40:32.035547 27671 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:32 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:34692
I20260808 23:40:32.039687 27368 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:40:32.041193 27368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:32.044013 27954 tablet_replica.cc:330] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: stopping tablet replica
I20260808 23:40:32.044433 27954 raft_consensus.cc:2229] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:40:32.045120 27954 raft_consensus.cc:2258] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:32.050271 27953 tablet_replica.cc:330] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: stopping tablet replica
I20260808 23:40:32.050567 27821 tablet_service.cc:1467] Processing CreateTablet for tablet 64f24a0d895b4111bd5412cb5d766f4d (DEFAULT_TABLE table=test-table [id=4e86dc2434714e98bf201e354fc18495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:32.050709 27516 tablet_service.cc:1467] Processing CreateTablet for tablet 64f24a0d895b4111bd5412cb5d766f4d (DEFAULT_TABLE table=test-table [id=4e86dc2434714e98bf201e354fc18495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:32.051086 27953 raft_consensus.cc:2229] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:32.051183 27821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64f24a0d895b4111bd5412cb5d766f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.051266 27516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64f24a0d895b4111bd5412cb5d766f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.051805 27953 raft_consensus.cc:2258] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:32.052683 27952 tablet_replica.cc:330] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: stopping tablet replica
I20260808 23:40:32.053092 27952 raft_consensus.cc:2229] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:32.053607 27952 raft_consensus.cc:2258] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:32.055886 27952 ts_tablet_manager.cc:1905] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:32.056789 27671 tablet_service.cc:1467] Processing CreateTablet for tablet 64f24a0d895b4111bd5412cb5d766f4d (DEFAULT_TABLE table=test-table [id=4e86dc2434714e98bf201e354fc18495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:32.057232 27671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64f24a0d895b4111bd5412cb5d766f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.058374 27928 tablet_bootstrap.cc:492] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: Bootstrap starting.
I20260808 23:40:32.059986 27928 tablet_bootstrap.cc:654] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.062228 27954 ts_tablet_manager.cc:1905] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:32.063637 27952 ts_tablet_manager.cc:1918] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:32.063791 27952 log.cc:1196] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/b2bea9f3cd0244c7b5bae6c456a3eaf9
I20260808 23:40:32.064343 27952 ts_tablet_manager.cc:1939] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P caee5f5f104b4f85a40c49876141376b: Deleting consensus metadata
I20260808 23:40:32.066236 27353 catalog_manager.cc:4841] TS caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865): tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 (table test-table [id=8c3c8c87d0dd4907a53da6c745cef9da]) successfully deleted
I20260808 23:40:32.066969 27953 ts_tablet_manager.cc:1905] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:32.068583 27928 tablet_bootstrap.cc:492] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: No bootstrap required, opened a new log
I20260808 23:40:32.068778 27928 ts_tablet_manager.cc:1397] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260808 23:40:32.069591 27928 raft_consensus.cc:348] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.069890 27928 raft_consensus.cc:374] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.070007 27928 raft_consensus.cc:729] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caee5f5f104b4f85a40c49876141376b, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.070305 27928 consensus_queue.cc:260] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.071029 27928 ts_tablet_manager.cc:1428] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 23:40:32.071636 27925 tablet_bootstrap.cc:492] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: Bootstrap starting.
I20260808 23:40:32.073412 27925 tablet_bootstrap.cc:654] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.073755 27953 ts_tablet_manager.cc:1918] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:32.073940 27953 log.cc:1196] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/b2bea9f3cd0244c7b5bae6c456a3eaf9
I20260808 23:40:32.074484 27953 ts_tablet_manager.cc:1939] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 277f0ee0121b49afb3213a37ca474631: Deleting consensus metadata
I20260808 23:40:32.076694 27351 catalog_manager.cc:4841] TS 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277): tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 (table test-table [id=8c3c8c87d0dd4907a53da6c745cef9da]) successfully deleted
I20260808 23:40:32.080765 27925 tablet_bootstrap.cc:492] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: No bootstrap required, opened a new log
I20260808 23:40:32.080988 27925 ts_tablet_manager.cc:1397] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260808 23:40:32.081815 27925 raft_consensus.cc:348] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.082134 27925 raft_consensus.cc:374] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.082410 27927 tablet_bootstrap.cc:492] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: Bootstrap starting.
I20260808 23:40:32.082263 27925 raft_consensus.cc:729] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277f0ee0121b49afb3213a37ca474631, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.082811 27925 consensus_queue.cc:260] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.083389 27925 ts_tablet_manager.cc:1428] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:40:32.084120 27927 tablet_bootstrap.cc:654] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.085422 27954 ts_tablet_manager.cc:1918] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:32.085615 27954 log.cc:1196] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/b2bea9f3cd0244c7b5bae6c456a3eaf9
I20260808 23:40:32.086234 27954 ts_tablet_manager.cc:1939] T b2bea9f3cd0244c7b5bae6c456a3eaf9 P 3a42ff2112fb48d493827466edf7b510: Deleting consensus metadata
I20260808 23:40:32.086762 27927 tablet_bootstrap.cc:492] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: No bootstrap required, opened a new log
I20260808 23:40:32.086928 27927 ts_tablet_manager.cc:1397] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 23:40:32.087785 27927 raft_consensus.cc:348] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.087990 27927 raft_consensus.cc:374] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.088089 27927 raft_consensus.cc:729] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a42ff2112fb48d493827466edf7b510, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.088124 27351 catalog_manager.cc:4841] TS 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297): tablet b2bea9f3cd0244c7b5bae6c456a3eaf9 (table test-table [id=8c3c8c87d0dd4907a53da6c745cef9da]) successfully deleted
I20260808 23:40:32.088352 27927 consensus_queue.cc:260] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.088935 27927 ts_tablet_manager.cc:1428] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:40:32.288172 27932 raft_consensus.cc:482] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:32.288463 27932 raft_consensus.cc:504] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.290422 27932 leader_election.cc:288] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297)
I20260808 23:40:32.298134 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.291527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:54655 (local address 127.18.186.130:35297)
0808 23:40:32.291709 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.291715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.291958 (+ 243us) server_negotiation.cc:407] Connection header received
0808 23:40:32.292181 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.292186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.292264 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.292331 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:32.292934 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.293897 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.295588 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.296727 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.296882 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.296971 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:32.297638 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 23:40:32.297780 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:40:32.298657 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.290622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:35277 (local address 127.18.186.131:40381)
0808 23:40:32.291157 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:32.291177 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:32.291325 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:32.292079 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:32.292083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:32.292111 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:32.292342 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.292354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.293871 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.293877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.295910 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.295926 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.297671 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.297677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.297848 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.297879 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:40:32.297967 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:40:32.298087 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":290,"threads_started":1}
I20260808 23:40:32.298935 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.291314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:35297 (local address 127.18.186.131:54655)
0808 23:40:32.291835 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:32.291854 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:32.292005 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:32.292537 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:32.292540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:32.292559 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:40:32.292777 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.292784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.294037 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.294043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.295375 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.295387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.296908 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.296915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.297429 (+ 514us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.297468 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 23:40:32.298404 (+ 936us) client_negotiation.cc:240] Negotiation successful
0808 23:40:32.298544 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":273,"threads_started":1}
I20260808 23:40:32.299022 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.290837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:40381 (local address 127.18.186.129:35277)
0808 23:40:32.291052 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.291059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.291760 (+ 701us) server_negotiation.cc:407] Connection header received
0808 23:40:32.291848 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.291854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.291933 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.292054 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:40:32.292508 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.293686 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.296082 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.297218 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.297415 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.297499 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:32.298587 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0808 23:40:32.298745 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 23:40:32.299952 27544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64f24a0d895b4111bd5412cb5d766f4d" candidate_uuid: "caee5f5f104b4f85a40c49876141376b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631" is_pre_election: true
I20260808 23:40:32.300020 27694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64f24a0d895b4111bd5412cb5d766f4d" candidate_uuid: "caee5f5f104b4f85a40c49876141376b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a42ff2112fb48d493827466edf7b510" is_pre_election: true
I20260808 23:40:32.300227 27544 raft_consensus.cc:2454] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caee5f5f104b4f85a40c49876141376b in term 0.
I20260808 23:40:32.300516 27694 raft_consensus.cc:2454] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caee5f5f104b4f85a40c49876141376b in term 0.
I20260808 23:40:32.300705 27768 leader_election.cc:302] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 277f0ee0121b49afb3213a37ca474631, caee5f5f104b4f85a40c49876141376b; no voters:
I20260808 23:40:32.301132 27932 raft_consensus.cc:2781] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:32.301259 27932 raft_consensus.cc:482] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:32.301362 27932 raft_consensus.cc:3037] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.303895 27932 raft_consensus.cc:504] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.304587 27932 leader_election.cc:288] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [CANDIDATE]: Term 1 election: Requested vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297)
I20260808 23:40:32.305032 27544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64f24a0d895b4111bd5412cb5d766f4d" candidate_uuid: "caee5f5f104b4f85a40c49876141376b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631"
I20260808 23:40:32.305137 27694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64f24a0d895b4111bd5412cb5d766f4d" candidate_uuid: "caee5f5f104b4f85a40c49876141376b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a42ff2112fb48d493827466edf7b510"
I20260808 23:40:32.305245 27544 raft_consensus.cc:3037] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.305362 27694 raft_consensus.cc:3037] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.307302 27544 raft_consensus.cc:2454] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caee5f5f104b4f85a40c49876141376b in term 1.
I20260808 23:40:32.307817 27768 leader_election.cc:302] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 277f0ee0121b49afb3213a37ca474631, caee5f5f104b4f85a40c49876141376b; no voters:
I20260808 23:40:32.308113 27932 raft_consensus.cc:2781] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:32.308650 27932 raft_consensus.cc:686] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 1 LEADER]: Becoming Leader. State: Replica: caee5f5f104b4f85a40c49876141376b, State: Running, Role: LEADER
I20260808 23:40:32.308990 27694 raft_consensus.cc:2454] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caee5f5f104b4f85a40c49876141376b in term 1.
I20260808 23:40:32.309394 27932 consensus_queue.cc:237] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.313010 27367 catalog_manager.cc:5495] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b reported cstate change: term changed from 0 to 1, leader changed from <none> to caee5f5f104b4f85a40c49876141376b (127.18.186.131). New cstate: current_term: 1 leader_uuid: "caee5f5f104b4f85a40c49876141376b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } health_report { overall_health: HEALTHY } } }
Aug 08 23:40:32 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232429, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.190@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:32.399894 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.386017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34631 (local address 127.0.0.1:51598)
0808 23:40:32.386642 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:32.386667 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:32.386834 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:32.387250 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:32.387254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:32.387969 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:32.388169 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.388178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.389209 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.389214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.390124 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.390135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.390808 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.390814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.390987 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.391725 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:32.391747 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:32.394161 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:32.397319 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:32.397331 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:32.397340 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:32.397726 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:32.398085 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:32.398091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:32.398094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:32.398248 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:32.398766 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:32.398776 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:32.398950 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 23:40:32.399113 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 23:40:32.399407 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":288,"threads_started":1}
I20260808 23:40:32.399948 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.386054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51598 (local address 127.18.186.190:34631)
0808 23:40:32.386996 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.387000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.387017 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:40:32.387055 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.387059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.387111 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.387224 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:32.388308 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.389064 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.390260 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.390651 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.390790 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.394353 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:32.394358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:32.394361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:32.394388 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:32.397187 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:32.397842 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:32.397850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:32.397856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:32.397982 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:32.398349 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:32.398357 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:32.398360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:32.398637 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:32.398799 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:32.399197 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 23:40:32.399585 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":773,"thread_start_us":626,"threads_started":1}
I20260808 23:40:32.405633 27367 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:51598:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 23:40:32.406692 27367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:32.414151 27671 tablet_service.cc:1467] Processing CreateTablet for tablet 105ed908a5a146778ba713ffe7db7bb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77e06f96da6d4198af0e2ab1fc87ade7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:40:32.414147 27516 tablet_service.cc:1467] Processing CreateTablet for tablet 105ed908a5a146778ba713ffe7db7bb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77e06f96da6d4198af0e2ab1fc87ade7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:40:32.414738 27671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105ed908a5a146778ba713ffe7db7bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.414757 27516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105ed908a5a146778ba713ffe7db7bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.416642 27821 tablet_service.cc:1467] Processing CreateTablet for tablet 105ed908a5a146778ba713ffe7db7bb1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77e06f96da6d4198af0e2ab1fc87ade7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:40:32.417145 27821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105ed908a5a146778ba713ffe7db7bb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.422616 27925 tablet_bootstrap.cc:492] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: Bootstrap starting.
I20260808 23:40:32.422647 27927 tablet_bootstrap.cc:492] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: Bootstrap starting.
I20260808 23:40:32.424423 27925 tablet_bootstrap.cc:654] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.424476 27927 tablet_bootstrap.cc:654] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.425552 27928 tablet_bootstrap.cc:492] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: Bootstrap starting.
I20260808 23:40:32.427279 27928 tablet_bootstrap.cc:654] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.431417 27928 tablet_bootstrap.cc:492] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: No bootstrap required, opened a new log
I20260808 23:40:32.431731 27928 ts_tablet_manager.cc:1397] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 23:40:32.433907 27928 raft_consensus.cc:348] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.434221 27927 tablet_bootstrap.cc:492] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: No bootstrap required, opened a new log
I20260808 23:40:32.434250 27928 raft_consensus.cc:374] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.434418 27927 ts_tablet_manager.cc:1397] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260808 23:40:32.434432 27928 raft_consensus.cc:729] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caee5f5f104b4f85a40c49876141376b, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.434800 27928 consensus_queue.cc:260] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.435258 27927 raft_consensus.cc:348] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.435297 27932 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.435559 27927 raft_consensus.cc:374] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.435623 27932 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:40:32.435657 27927 raft_consensus.cc:729] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a42ff2112fb48d493827466edf7b510, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.435990 27927 consensus_queue.cc:260] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.436748 27928 ts_tablet_manager.cc:1428] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 23:40:32.437098 27933 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.437374 27933 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:40:32.439527 27927 ts_tablet_manager.cc:1428] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260808 23:40:32.440291 27925 tablet_bootstrap.cc:492] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: No bootstrap required, opened a new log
I20260808 23:40:32.440478 27925 ts_tablet_manager.cc:1397] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.000s
I20260808 23:40:32.441170 27925 raft_consensus.cc:348] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.441422 27925 raft_consensus.cc:374] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.441498 27925 raft_consensus.cc:729] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277f0ee0121b49afb3213a37ca474631, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.441731 27925 consensus_queue.cc:260] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.442343 27925 ts_tablet_manager.cc:1428] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:40:32.442239 27931 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.442644 27931 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:40:32.480193 27933 raft_consensus.cc:482] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:32.480409 27933 raft_consensus.cc:504] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.481092 27933 leader_election.cc:288] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865)
I20260808 23:40:32.481626 27544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105ed908a5a146778ba713ffe7db7bb1" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631" is_pre_election: true
I20260808 23:40:32.481652 27851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105ed908a5a146778ba713ffe7db7bb1" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caee5f5f104b4f85a40c49876141376b" is_pre_election: true
I20260808 23:40:32.481904 27544 raft_consensus.cc:2454] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 0.
I20260808 23:40:32.481912 27851 raft_consensus.cc:2454] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 0.
I20260808 23:40:32.482362 27622 leader_election.cc:302] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a42ff2112fb48d493827466edf7b510, caee5f5f104b4f85a40c49876141376b; no voters:
I20260808 23:40:32.482654 27933 raft_consensus.cc:2781] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:32.482788 27933 raft_consensus.cc:482] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:32.482914 27933 raft_consensus.cc:3037] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.485105 27933 raft_consensus.cc:504] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.485670 27933 leader_election.cc:288] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 election: Requested vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865)
I20260808 23:40:32.486068 27544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105ed908a5a146778ba713ffe7db7bb1" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631"
I20260808 23:40:32.486218 27851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105ed908a5a146778ba713ffe7db7bb1" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caee5f5f104b4f85a40c49876141376b"
I20260808 23:40:32.486328 27544 raft_consensus.cc:3037] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.486469 27851 raft_consensus.cc:3037] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.488473 27544 raft_consensus.cc:2454] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 1.
I20260808 23:40:32.488629 27851 raft_consensus.cc:2454] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 1.
I20260808 23:40:32.489049 27620 leader_election.cc:302] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 277f0ee0121b49afb3213a37ca474631, 3a42ff2112fb48d493827466edf7b510; no voters:
I20260808 23:40:32.489388 27933 raft_consensus.cc:2781] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:32.489550 27933 raft_consensus.cc:686] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [term 1 LEADER]: Becoming Leader. State: Replica: 3a42ff2112fb48d493827466edf7b510, State: Running, Role: LEADER
I20260808 23:40:32.489838 27933 consensus_queue.cc:237] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.491328 27949 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: TxnStatusTablet state changed. Reason: New leader 3a42ff2112fb48d493827466edf7b510. Latest consensus state: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.491652 27949 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:40:32.492277 27974 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 23:40:32.492486 27974 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 23:40:32.492751 27367 catalog_manager.cc:5495] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a42ff2112fb48d493827466edf7b510 (127.18.186.130). New cstate: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } health_report { overall_health: UNKNOWN } } }
I20260808 23:40:32.518776 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.512771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.18.186.130:35297)
0808 23:40:32.512948 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.512953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.513264 (+ 311us) server_negotiation.cc:407] Connection header received
0808 23:40:32.513526 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.513529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.513593 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.513667 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:32.514118 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.514907 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.516997 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.517515 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.517616 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.518160 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:32.518258 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:32.518327 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:32.518502 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 23:40:32.518567 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 23:40:32.518967 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.512795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:35297 (local address 127.0.0.1:34708)
0808 23:40:32.513173 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:32.513193 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:32.513354 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:32.513726 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:32.513731 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:32.513744 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:40:32.513949 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.513963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.515043 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.515049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.516718 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.516734 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.517671 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.517678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.517883 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.517971 (+ 88us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:40:32.518361 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:40:32.518382 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 23:40:32.518472 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:40:32.518628 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260808 23:40:32.522755 27851 raft_consensus.cc:1261] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Refusing update from remote peer 3a42ff2112fb48d493827466edf7b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:32.522770 27544 raft_consensus.cc:1261] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Refusing update from remote peer 3a42ff2112fb48d493827466edf7b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:32.523363 27949 consensus_queue.cc:1035] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:32.523676 27933 consensus_queue.cc:1035] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:32.533564 27931 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: TxnStatusTablet state changed. Reason: New leader 3a42ff2112fb48d493827466edf7b510. Latest consensus state: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.533870 27931 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:40:32.535632 27932 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: TxnStatusTablet state changed. Reason: New leader 3a42ff2112fb48d493827466edf7b510. Latest consensus state: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.535946 27932 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:40:32.537691 27978 mvcc.cc:205] Tried to move back new op lower bound from 7316408043607240704 to 7316408043480547328. Current Snapshot: MvccSnapshot[applied={T|T < 7316408043607240704}]
I20260808 23:40:32.544999 27931 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.545250 27931 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P 277f0ee0121b49afb3213a37ca474631: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:40:32.553120 27949 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.553421 27949 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:40:32.553275 27933 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.553573 27933 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P 3a42ff2112fb48d493827466edf7b510: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:40:32.555502 27932 tablet_replica.cc:439] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } } }
I20260808 23:40:32.555800 27932 tablet_replica.cc:442] T 105ed908a5a146778ba713ffe7db7bb1 P caee5f5f104b4f85a40c49876141376b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:40:32.570303 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.558022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.18.186.131:42865)
0808 23:40:32.564586 (+ 6564us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.564592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.564614 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:40:32.564713 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.564719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.564831 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.565035 (+ 204us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:32.565563 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.566498 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.568015 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.568487 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.568618 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.569088 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:32.569688 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:32.569843 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:32.569915 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 23:40:32.569994 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6436,"thread_start_us":6313,"threads_started":1}
I20260808 23:40:32.586447 27991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.580149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:35297 (local address 127.18.186.131:54187)
0808 23:40:32.580818 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:32.580834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:32.580943 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:32.581285 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:32.581289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:32.581298 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:40:32.581572 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.581579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.583027 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.583030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.584148 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.584159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.585197 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.585201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.585396 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.585451 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:40:32.585856 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:40:32.585864 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:40:32.585922 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 23:40:32.586014 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":359,"threads_started":1}
I20260808 23:40:32.586510 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.580196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:54187 (local address 127.18.186.130:35297)
0808 23:40:32.580466 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.580473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.580916 (+ 443us) server_negotiation.cc:407] Connection header received
0808 23:40:32.581066 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.581071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.581167 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.581260 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:32.581719 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.582913 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.584287 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.585046 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.585169 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.585612 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:32.585759 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:32.585850 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:32.585956 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 23:40:32.586038 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260808 23:40:32.595568 27543 raft_consensus.cc:1261] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Refusing update from remote peer caee5f5f104b4f85a40c49876141376b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:32.595806 27694 raft_consensus.cc:1261] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Refusing update from remote peer caee5f5f104b4f85a40c49876141376b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:32.596555 27961 consensus_queue.cc:1035] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:32.596909 27932 consensus_queue.cc:1035] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [LEADER]: Connected to new peer: Peer: permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:32.642124 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.634720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:60731 (local address 127.18.186.131:42865)
0808 23:40:32.634934 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.634942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.635776 (+ 834us) server_negotiation.cc:407] Connection header received
0808 23:40:32.635996 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.636001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.636105 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.636198 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:32.637005 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.637835 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.640091 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.640731 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.640827 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.641482 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:32.641579 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:32.641650 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:32.641801 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 23:40:32.641900 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:40:32.642310 27996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.634741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:42865 (local address 127.18.186.130:60731)
0808 23:40:32.635686 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:32.635708 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:32.635853 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:32.636255 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:32.636260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:32.636275 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:40:32.636668 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.636681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.637964 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.637970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.639915 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:32.639930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.641016 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.641022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:32.641208 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.641275 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:40:32.641679 (+ 404us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:40:32.641692 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 23:40:32.641782 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:40:32.641932 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":360,"threads_started":1}
I20260808 23:40:32.692013 27367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:32.692278 27367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:32.696005 27367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: Sending DeleteTablet for 3 replicas of tablet 64f24a0d895b4111bd5412cb5d766f4d
I20260808 23:40:32.696977 27516 tablet_service.cc:1514] Processing DeleteTablet for tablet 64f24a0d895b4111bd5412cb5d766f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:32 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:51058
I20260808 23:40:32.697233 27671 tablet_service.cc:1514] Processing DeleteTablet for tablet 64f24a0d895b4111bd5412cb5d766f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:32 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:34692
I20260808 23:40:32.697654 27821 tablet_service.cc:1514] Processing DeleteTablet for tablet 64f24a0d895b4111bd5412cb5d766f4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:32 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:59266
I20260808 23:40:32.697975 28001 tablet_replica.cc:330] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: stopping tablet replica
I20260808 23:40:32.698227 28001 raft_consensus.cc:2229] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:32.698555 28001 raft_consensus.cc:2258] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:32.708894 28001 ts_tablet_manager.cc:1905] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:32.714099 28002 tablet_replica.cc:330] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: stopping tablet replica
I20260808 23:40:32.714322 28002 raft_consensus.cc:2229] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:40:32.714962 28002 raft_consensus.cc:2258] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:32.718901 28000 tablet_replica.cc:330] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: stopping tablet replica
I20260808 23:40:32.719151 28000 raft_consensus.cc:2229] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:32.719533 28000 raft_consensus.cc:2258] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:32.720717 28000 ts_tablet_manager.cc:1905] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:32.721185 28001 ts_tablet_manager.cc:1918] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:40:32.721338 28001 log.cc:1196] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/64f24a0d895b4111bd5412cb5d766f4d
I20260808 23:40:32.721864 28001 ts_tablet_manager.cc:1939] T 64f24a0d895b4111bd5412cb5d766f4d P 3a42ff2112fb48d493827466edf7b510: Deleting consensus metadata
I20260808 23:40:32.723037 27351 catalog_manager.cc:4841] TS 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297): tablet 64f24a0d895b4111bd5412cb5d766f4d (table test-table [id=4e86dc2434714e98bf201e354fc18495]) successfully deleted
I20260808 23:40:32.726248 28002 ts_tablet_manager.cc:1905] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:32.733151 28000 ts_tablet_manager.cc:1918] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:40:32.733157 28002 ts_tablet_manager.cc:1918] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:40:32.733289 28002 log.cc:1196] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/64f24a0d895b4111bd5412cb5d766f4d
I20260808 23:40:32.733417 28000 log.cc:1196] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/64f24a0d895b4111bd5412cb5d766f4d
I20260808 23:40:32.733819 28002 ts_tablet_manager.cc:1939] T 64f24a0d895b4111bd5412cb5d766f4d P caee5f5f104b4f85a40c49876141376b: Deleting consensus metadata
I20260808 23:40:32.733969 28000 ts_tablet_manager.cc:1939] T 64f24a0d895b4111bd5412cb5d766f4d P 277f0ee0121b49afb3213a37ca474631: Deleting consensus metadata
I20260808 23:40:32.735056 27351 catalog_manager.cc:4841] TS 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277): tablet 64f24a0d895b4111bd5412cb5d766f4d (table test-table [id=4e86dc2434714e98bf201e354fc18495]) successfully deleted
I20260808 23:40:32.735056 27353 catalog_manager.cc:4841] TS caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865): tablet 64f24a0d895b4111bd5412cb5d766f4d (table test-table [id=4e86dc2434714e98bf201e354fc18495]) successfully deleted
Aug 08 23:40:32 dist-test-slave-5k54 krb5kdc[27321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:40:32 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:32 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:40:32.798988 27969 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:32.785710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.18.186.190:34631)
0808 23:40:32.785927 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.785933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.786468 (+ 535us) server_negotiation.cc:407] Connection header received
0808 23:40:32.786651 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.786657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.786738 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.786839 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:32.788688 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.789781 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.791646 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.792336 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.792467 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.798709 (+ 6242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51614: BlockingRecv error: recv got EOF from 127.0.0.1:51614 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 23:40:32 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM
I20260808 23:40:32.821421 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.803096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.18.186.190:34631)
0808 23:40:32.803271 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:32.803277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:32.803948 (+ 671us) server_negotiation.cc:407] Connection header received
0808 23:40:32.804114 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:32.804120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:32.804207 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:32.804331 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:32.805835 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.806988 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.808903 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:32.809563 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:32.809695 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:32.814615 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:32.814622 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:32.814627 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:32.814671 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:32.818260 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:32.819120 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:32.819126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:32.819130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:32.819301 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:32.819824 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:32.819829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:32.819833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:32.820181 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:32.820455 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:32.820787 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 23:40:32.821047 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:40:32.825656 27367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:40:32.826429 27367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:32.834450 27821 tablet_service.cc:1467] Processing CreateTablet for tablet c48e5eb4f914477ca9e708c53de464d2 (DEFAULT_TABLE table=test-table [id=8bc78864233a45f2a0641c6344ba4188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:32.834530 27516 tablet_service.cc:1467] Processing CreateTablet for tablet c48e5eb4f914477ca9e708c53de464d2 (DEFAULT_TABLE table=test-table [id=8bc78864233a45f2a0641c6344ba4188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:32.834954 27821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48e5eb4f914477ca9e708c53de464d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.835019 27516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48e5eb4f914477ca9e708c53de464d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.835886 27671 tablet_service.cc:1467] Processing CreateTablet for tablet c48e5eb4f914477ca9e708c53de464d2 (DEFAULT_TABLE table=test-table [id=8bc78864233a45f2a0641c6344ba4188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:32.836320 27671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48e5eb4f914477ca9e708c53de464d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:32.841756 27925 tablet_bootstrap.cc:492] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: Bootstrap starting.
I20260808 23:40:32.843506 27925 tablet_bootstrap.cc:654] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.850984 27928 tablet_bootstrap.cc:492] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: Bootstrap starting.
I20260808 23:40:32.852777 27928 tablet_bootstrap.cc:654] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.857856 27927 tablet_bootstrap.cc:492] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: Bootstrap starting.
I20260808 23:40:32.859743 27927 tablet_bootstrap.cc:654] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:32.862097 27928 tablet_bootstrap.cc:492] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: No bootstrap required, opened a new log
I20260808 23:40:32.862157 27927 tablet_bootstrap.cc:492] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: No bootstrap required, opened a new log
I20260808 23:40:32.862238 27928 ts_tablet_manager.cc:1397] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260808 23:40:32.862334 27927 ts_tablet_manager.cc:1397] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 23:40:32.863082 27928 raft_consensus.cc:348] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.863358 27928 raft_consensus.cc:374] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.863195 27927 raft_consensus.cc:348] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.863463 27928 raft_consensus.cc:729] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caee5f5f104b4f85a40c49876141376b, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.863495 27927 raft_consensus.cc:374] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.863619 27927 raft_consensus.cc:729] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a42ff2112fb48d493827466edf7b510, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.863724 27928 consensus_queue.cc:260] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.863862 27927 consensus_queue.cc:260] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.864287 27928 ts_tablet_manager.cc:1428] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:40:32.864434 27927 ts_tablet_manager.cc:1428] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:40:32.877264 27925 tablet_bootstrap.cc:492] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: No bootstrap required, opened a new log
I20260808 23:40:32.877462 27925 ts_tablet_manager.cc:1397] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.012s
I20260808 23:40:32.878351 27925 raft_consensus.cc:348] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.878628 27925 raft_consensus.cc:374] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:32.878720 27925 raft_consensus.cc:729] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277f0ee0121b49afb3213a37ca474631, State: Initialized, Role: FOLLOWER
I20260808 23:40:32.878958 27925 consensus_queue.cc:260] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.879555 27925 ts_tablet_manager.cc:1428] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:40:32.952433 27949 raft_consensus.cc:482] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:32.952677 27949 raft_consensus.cc:504] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.953512 27949 leader_election.cc:288] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865)
I20260808 23:40:32.954089 27543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48e5eb4f914477ca9e708c53de464d2" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631" is_pre_election: true
I20260808 23:40:32.954207 27851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48e5eb4f914477ca9e708c53de464d2" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caee5f5f104b4f85a40c49876141376b" is_pre_election: true
I20260808 23:40:32.954375 27543 raft_consensus.cc:2454] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 0.
I20260808 23:40:32.954463 27851 raft_consensus.cc:2454] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 0.
I20260808 23:40:32.954922 27622 leader_election.cc:302] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a42ff2112fb48d493827466edf7b510, caee5f5f104b4f85a40c49876141376b; no voters:
I20260808 23:40:32.955250 27949 raft_consensus.cc:2781] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:32.955353 27949 raft_consensus.cc:482] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:32.955475 27949 raft_consensus.cc:3037] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.958767 27949 raft_consensus.cc:504] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.959972 27543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48e5eb4f914477ca9e708c53de464d2" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631"
I20260808 23:40:32.960057 27851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48e5eb4f914477ca9e708c53de464d2" candidate_uuid: "3a42ff2112fb48d493827466edf7b510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caee5f5f104b4f85a40c49876141376b"
I20260808 23:40:32.960183 27543 raft_consensus.cc:3037] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.960243 27851 raft_consensus.cc:3037] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:32.963120 27543 raft_consensus.cc:2454] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 1.
I20260808 23:40:32.963531 27851 raft_consensus.cc:2454] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a42ff2112fb48d493827466edf7b510 in term 1.
I20260808 23:40:32.963677 27620 leader_election.cc:302] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 277f0ee0121b49afb3213a37ca474631, 3a42ff2112fb48d493827466edf7b510; no voters:
I20260808 23:40:32.963994 27998 raft_consensus.cc:2781] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:32.964099 27949 leader_election.cc:288] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [CANDIDATE]: Term 1 election: Requested vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865)
I20260808 23:40:32.964175 27998 raft_consensus.cc:686] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 1 LEADER]: Becoming Leader. State: Replica: 3a42ff2112fb48d493827466edf7b510, State: Running, Role: LEADER
I20260808 23:40:32.964525 27998 consensus_queue.cc:237] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:32.968055 27367 catalog_manager.cc:5495] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a42ff2112fb48d493827466edf7b510 (127.18.186.130). New cstate: current_term: 1 leader_uuid: "3a42ff2112fb48d493827466edf7b510" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } health_report { overall_health: UNKNOWN } } }
I20260808 23:40:33.005710 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:32.999881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.18.186.130:35297)
0808 23:40:33.000442 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:33.000447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:33.000461 (+ 14us) server_negotiation.cc:407] Connection header received
0808 23:40:33.000517 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:33.000520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:33.000582 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:33.000652 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:33.001148 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.001929 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.003941 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.004339 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.004437 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:33.004995 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:33.005108 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:33.005175 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:33.005336 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 23:40:33.005402 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447}
I20260808 23:40:33.008783 27543 raft_consensus.cc:1261] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Refusing update from remote peer 3a42ff2112fb48d493827466edf7b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:33.008857 27851 raft_consensus.cc:1261] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Refusing update from remote peer 3a42ff2112fb48d493827466edf7b510: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:33.009492 27998 consensus_queue.cc:1035] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:33.009796 27933 consensus_queue.cc:1035] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:33.028584 27367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:33.028836 27367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:33.031704 27367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: Sending DeleteTablet for 3 replicas of tablet c48e5eb4f914477ca9e708c53de464d2
I20260808 23:40:33.032586 27671 tablet_service.cc:1514] Processing DeleteTablet for tablet c48e5eb4f914477ca9e708c53de464d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:33 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:34692
I20260808 23:40:33.032560 27516 tablet_service.cc:1514] Processing DeleteTablet for tablet c48e5eb4f914477ca9e708c53de464d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:33 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:51058
I20260808 23:40:33.032846 28000 tablet_replica.cc:330] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: stopping tablet replica
I20260808 23:40:33.032881 28001 tablet_replica.cc:330] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: stopping tablet replica
I20260808 23:40:33.033056 28000 raft_consensus.cc:2229] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:33.033088 28001 raft_consensus.cc:2229] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:40:33.033298 28000 raft_consensus.cc:2258] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:33.033452 28001 raft_consensus.cc:2258] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:33.034055 27821 tablet_service.cc:1514] Processing DeleteTablet for tablet c48e5eb4f914477ca9e708c53de464d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:33 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:59266
I20260808 23:40:33.034380 28002 tablet_replica.cc:330] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: stopping tablet replica
I20260808 23:40:33.034593 28002 raft_consensus.cc:2229] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:33.034802 28002 raft_consensus.cc:2258] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:33.035058 28001 ts_tablet_manager.cc:1905] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:33.035101 28000 ts_tablet_manager.cc:1905] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:33.036180 28002 ts_tablet_manager.cc:1905] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:33.041878 28001 ts_tablet_manager.cc:1918] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:33.041968 28000 ts_tablet_manager.cc:1918] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:33.042081 28001 log.cc:1196] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/c48e5eb4f914477ca9e708c53de464d2
I20260808 23:40:33.042120 28000 log.cc:1196] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/c48e5eb4f914477ca9e708c53de464d2
I20260808 23:40:33.042546 28000 ts_tablet_manager.cc:1939] T c48e5eb4f914477ca9e708c53de464d2 P 277f0ee0121b49afb3213a37ca474631: Deleting consensus metadata
I20260808 23:40:33.042546 28001 ts_tablet_manager.cc:1939] T c48e5eb4f914477ca9e708c53de464d2 P 3a42ff2112fb48d493827466edf7b510: Deleting consensus metadata
I20260808 23:40:33.043730 27351 catalog_manager.cc:4841] TS 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297): tablet c48e5eb4f914477ca9e708c53de464d2 (table test-table [id=8bc78864233a45f2a0641c6344ba4188]) successfully deleted
I20260808 23:40:33.043818 28002 ts_tablet_manager.cc:1918] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:40:33.043980 28002 log.cc:1196] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/c48e5eb4f914477ca9e708c53de464d2
I20260808 23:40:33.044493 28002 ts_tablet_manager.cc:1939] T c48e5eb4f914477ca9e708c53de464d2 P caee5f5f104b4f85a40c49876141376b: Deleting consensus metadata
I20260808 23:40:33.045379 27351 catalog_manager.cc:4841] TS 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277): tablet c48e5eb4f914477ca9e708c53de464d2 (table test-table [id=8bc78864233a45f2a0641c6344ba4188]) successfully deleted
I20260808 23:40:33.045708 27353 catalog_manager.cc:4841] TS caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865): tablet c48e5eb4f914477ca9e708c53de464d2 (table test-table [id=8bc78864233a45f2a0641c6344ba4188]) successfully deleted
I20260808 23:40:33.045969 27367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:40:33.046607 27367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:40:33.054145 27516 tablet_service.cc:1467] Processing CreateTablet for tablet 30f8f5d49d0748fbb49c84cbd54cafbe (DEFAULT_TABLE table=test-table [id=75dbe0aa57ae4490b2eecd40a8c34ca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:33.054209 27821 tablet_service.cc:1467] Processing CreateTablet for tablet 30f8f5d49d0748fbb49c84cbd54cafbe (DEFAULT_TABLE table=test-table [id=75dbe0aa57ae4490b2eecd40a8c34ca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:33.054601 27516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f8f5d49d0748fbb49c84cbd54cafbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:33.054621 27821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f8f5d49d0748fbb49c84cbd54cafbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:33.059032 27671 tablet_service.cc:1467] Processing CreateTablet for tablet 30f8f5d49d0748fbb49c84cbd54cafbe (DEFAULT_TABLE table=test-table [id=75dbe0aa57ae4490b2eecd40a8c34ca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:40:33.059507 27671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f8f5d49d0748fbb49c84cbd54cafbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:33.061218 27925 tablet_bootstrap.cc:492] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: Bootstrap starting.
I20260808 23:40:33.062794 27925 tablet_bootstrap.cc:654] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:33.072243 27927 tablet_bootstrap.cc:492] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: Bootstrap starting.
I20260808 23:40:33.073087 27928 tablet_bootstrap.cc:492] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: Bootstrap starting.
I20260808 23:40:33.074303 27927 tablet_bootstrap.cc:654] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:33.075115 27928 tablet_bootstrap.cc:654] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:33.078207 27925 tablet_bootstrap.cc:492] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: No bootstrap required, opened a new log
I20260808 23:40:33.079236 27928 tablet_bootstrap.cc:492] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: No bootstrap required, opened a new log
I20260808 23:40:33.079528 27925 ts_tablet_manager.cc:1397] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.012s
I20260808 23:40:33.079579 27928 ts_tablet_manager.cc:1397] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260808 23:40:33.080680 27925 raft_consensus.cc:348] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.080974 27925 raft_consensus.cc:374] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:33.081087 27925 raft_consensus.cc:729] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277f0ee0121b49afb3213a37ca474631, State: Initialized, Role: FOLLOWER
I20260808 23:40:33.081128 27928 raft_consensus.cc:348] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.081378 27925 consensus_queue.cc:260] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.081549 27928 raft_consensus.cc:374] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:33.081748 27928 raft_consensus.cc:729] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caee5f5f104b4f85a40c49876141376b, State: Initialized, Role: FOLLOWER
I20260808 23:40:33.081943 27925 ts_tablet_manager.cc:1428] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 23:40:33.082105 27928 consensus_queue.cc:260] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.085717 27927 tablet_bootstrap.cc:492] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: No bootstrap required, opened a new log
I20260808 23:40:33.085899 27927 ts_tablet_manager.cc:1397] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260808 23:40:33.086097 27928 ts_tablet_manager.cc:1428] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260808 23:40:33.086901 27927 raft_consensus.cc:348] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.087167 27927 raft_consensus.cc:374] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:33.087257 27927 raft_consensus.cc:729] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a42ff2112fb48d493827466edf7b510, State: Initialized, Role: FOLLOWER
I20260808 23:40:33.087563 27927 consensus_queue.cc:260] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.088225 27927 ts_tablet_manager.cc:1428] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:40:33.147197 27994 raft_consensus.cc:482] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:40:33.147454 27994 raft_consensus.cc:504] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.148164 27994 leader_election.cc:288] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297)
I20260808 23:40:33.148811 27694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f8f5d49d0748fbb49c84cbd54cafbe" candidate_uuid: "caee5f5f104b4f85a40c49876141376b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a42ff2112fb48d493827466edf7b510" is_pre_election: true
I20260808 23:40:33.148880 27543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f8f5d49d0748fbb49c84cbd54cafbe" candidate_uuid: "caee5f5f104b4f85a40c49876141376b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631" is_pre_election: true
I20260808 23:40:33.149091 27694 raft_consensus.cc:2454] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caee5f5f104b4f85a40c49876141376b in term 0.
I20260808 23:40:33.149199 27543 raft_consensus.cc:2454] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caee5f5f104b4f85a40c49876141376b in term 0.
I20260808 23:40:33.149585 27768 leader_election.cc:302] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a42ff2112fb48d493827466edf7b510, caee5f5f104b4f85a40c49876141376b; no voters:
I20260808 23:40:33.149930 27994 raft_consensus.cc:2781] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:40:33.150058 27994 raft_consensus.cc:482] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:40:33.150187 27994 raft_consensus.cc:3037] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:33.152444 27994 raft_consensus.cc:504] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.153059 27994 leader_election.cc:288] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [CANDIDATE]: Term 1 election: Requested vote from peers 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277), 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297)
I20260808 23:40:33.153561 27694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f8f5d49d0748fbb49c84cbd54cafbe" candidate_uuid: "caee5f5f104b4f85a40c49876141376b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a42ff2112fb48d493827466edf7b510"
I20260808 23:40:33.153543 27543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f8f5d49d0748fbb49c84cbd54cafbe" candidate_uuid: "caee5f5f104b4f85a40c49876141376b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "277f0ee0121b49afb3213a37ca474631"
I20260808 23:40:33.153753 27694 raft_consensus.cc:3037] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:33.153771 27543 raft_consensus.cc:3037] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:33.156657 27543 raft_consensus.cc:2454] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caee5f5f104b4f85a40c49876141376b in term 1.
I20260808 23:40:33.156672 27694 raft_consensus.cc:2454] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caee5f5f104b4f85a40c49876141376b in term 1.
I20260808 23:40:33.157061 27768 leader_election.cc:302] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 277f0ee0121b49afb3213a37ca474631, caee5f5f104b4f85a40c49876141376b; no voters:
I20260808 23:40:33.157370 27994 raft_consensus.cc:2781] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:33.157546 27994 raft_consensus.cc:686] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 1 LEADER]: Becoming Leader. State: Replica: caee5f5f104b4f85a40c49876141376b, State: Running, Role: LEADER
I20260808 23:40:33.157850 27994 consensus_queue.cc:237] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } }
I20260808 23:40:33.160974 27366 catalog_manager.cc:5495] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b reported cstate change: term changed from 0 to 1, leader changed from <none> to caee5f5f104b4f85a40c49876141376b (127.18.186.131). New cstate: current_term: 1 leader_uuid: "caee5f5f104b4f85a40c49876141376b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caee5f5f104b4f85a40c49876141376b" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 42865 } health_report { overall_health: HEALTHY } } }
I20260808 23:40:33.183285 28029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:33.177702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.18.186.131:42865)
0808 23:40:33.178396 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:33.178408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:33.178428 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:40:33.178493 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:33.178498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:33.178576 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:33.178699 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:40:33.179206 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.180130 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.181443 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.181913 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.182054 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:33.182517 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:40:33.182627 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:40:33.182728 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:33.182887 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 23:40:33.182962 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":369,"threads_started":1}
I20260808 23:40:33.192344 27543 raft_consensus.cc:1261] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Refusing update from remote peer caee5f5f104b4f85a40c49876141376b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:33.192386 27694 raft_consensus.cc:1261] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Refusing update from remote peer caee5f5f104b4f85a40c49876141376b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:40:33.192903 27994 consensus_queue.cc:1035] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a42ff2112fb48d493827466edf7b510" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 35297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:33.193197 27961 consensus_queue.cc:1035] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [LEADER]: Connected to new peer: Peer: permanent_uuid: "277f0ee0121b49afb3213a37ca474631" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:40:33.197986 27977 mvcc.cc:205] Tried to move back new op lower bound from 7316408046351331328 to 7316408046217236480. Current Snapshot: MvccSnapshot[applied={T|T < 7316408046351331328}]
I20260808 23:40:33.262425 27366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:33.262718 27366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:40:33.265641 27366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a772a0fbd3624e51a85ec20e46e2cabf: Sending DeleteTablet for 3 replicas of tablet 30f8f5d49d0748fbb49c84cbd54cafbe
I20260808 23:40:33.266498 27516 tablet_service.cc:1514] Processing DeleteTablet for tablet 30f8f5d49d0748fbb49c84cbd54cafbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:33 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:51058
I20260808 23:40:33.266592 27671 tablet_service.cc:1514] Processing DeleteTablet for tablet 30f8f5d49d0748fbb49c84cbd54cafbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:33 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:34692
I20260808 23:40:33.266817 28000 tablet_replica.cc:330] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: stopping tablet replica
I20260808 23:40:33.266922 28001 tablet_replica.cc:330] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: stopping tablet replica
I20260808 23:40:33.267015 28000 raft_consensus.cc:2229] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:33.267136 28001 raft_consensus.cc:2229] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:40:33.267235 28000 raft_consensus.cc:2258] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:33.267237 27821 tablet_service.cc:1514] Processing DeleteTablet for tablet 30f8f5d49d0748fbb49c84cbd54cafbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:40:33 UTC) from {username='oryx', principal='oryx/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:59266
I20260808 23:40:33.267349 28001 raft_consensus.cc:2258] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:33.267576 28002 tablet_replica.cc:330] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: stopping tablet replica
I20260808 23:40:33.267779 28002 raft_consensus.cc:2229] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:40:33.268178 28002 raft_consensus.cc:2258] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:40:33.268232 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 27446
I20260808 23:40:33.268528 28000 ts_tablet_manager.cc:1905] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:33.270624 28001 ts_tablet_manager.cc:1905] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:33.277872 28000 ts_tablet_manager.cc:1918] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:40:33.278074 28000 log.cc:1196] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/30f8f5d49d0748fbb49c84cbd54cafbe
I20260808 23:40:33.278602 28000 ts_tablet_manager.cc:1939] T 30f8f5d49d0748fbb49c84cbd54cafbe P 277f0ee0121b49afb3213a37ca474631: Deleting consensus metadata
I20260808 23:40:33.280016 27351 catalog_manager.cc:4841] TS 277f0ee0121b49afb3213a37ca474631 (127.18.186.129:35277): tablet 30f8f5d49d0748fbb49c84cbd54cafbe (table test-table [id=75dbe0aa57ae4490b2eecd40a8c34ca0]) successfully deleted
I20260808 23:40:33.280902 28002 ts_tablet_manager.cc:1905] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:40:33.281399 28001 ts_tablet_manager.cc:1918] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:40:33.281562 28001 log.cc:1196] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/30f8f5d49d0748fbb49c84cbd54cafbe
I20260808 23:40:33.282051 28001 ts_tablet_manager.cc:1939] T 30f8f5d49d0748fbb49c84cbd54cafbe P 3a42ff2112fb48d493827466edf7b510: Deleting consensus metadata
I20260808 23:40:33.282974 27351 catalog_manager.cc:4841] TS 3a42ff2112fb48d493827466edf7b510 (127.18.186.130:35297): tablet 30f8f5d49d0748fbb49c84cbd54cafbe (table test-table [id=75dbe0aa57ae4490b2eecd40a8c34ca0]) successfully deleted
I20260808 23:40:33.287214 28002 ts_tablet_manager.cc:1918] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:40:33.287358 28002 log.cc:1196] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/30f8f5d49d0748fbb49c84cbd54cafbe
I20260808 23:40:33.287809 28002 ts_tablet_manager.cc:1939] T 30f8f5d49d0748fbb49c84cbd54cafbe P caee5f5f104b4f85a40c49876141376b: Deleting consensus metadata
I20260808 23:40:33.289006 27353 catalog_manager.cc:4841] TS caee5f5f104b4f85a40c49876141376b (127.18.186.131:42865): tablet 30f8f5d49d0748fbb49c84cbd54cafbe (table test-table [id=75dbe0aa57ae4490b2eecd40a8c34ca0]) successfully deleted
Aug 08 23:40:33 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.129@KRBTEST.COM
I20260808 23:40:33.297066 28037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:33.269538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51070 (local address 127.18.186.129:35277)
0808 23:40:33.280327 (+ 10789us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:33.280331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:33.281706 (+ 1375us) server_negotiation.cc:407] Connection header received
0808 23:40:33.281894 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:33.281900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:33.281985 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:33.282117 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:33.283793 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.284542 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.286439 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.286983 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.287082 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:33.291402 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:33.291417 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:33.291484 (+ 67us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:33.291537 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:33.294472 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:33.295097 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:33.295105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:33.295107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:33.295262 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:33.295741 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:33.295745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:33.295747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:33.296096 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:33.296317 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:33.296531 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 23:40:33.296695 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10657,"thread_start_us":10545,"threads_started":1}
I20260808 23:40:33.297684 27591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:33.394083 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 27446
I20260808 23:40:33.415212 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 27600
Aug 08 23:40:33 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.130@KRBTEST.COM
I20260808 23:40:33.429816 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:33.416057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.18.186.130:35297)
0808 23:40:33.416237 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:33.416242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:33.416261 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:40:33.416394 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:33.416398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:33.416475 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:33.416557 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:33.417765 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.418471 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.419727 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.420140 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.420230 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:33.424387 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:33.424409 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:33.424425 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:33.424462 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:33.427379 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:33.428017 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:33.428024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:33.428026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:33.428157 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:33.428481 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:33.428484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:33.428486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:33.428800 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:33.429024 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:33.429251 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 23:40:33.429415 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:40:33.430465 27741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:33.583179 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 27600
I20260808 23:40:33.611017 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 27750
Aug 08 23:40:33 dist-test-slave-5k54 krb5kdc[27321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.186.131@KRBTEST.COM
I20260808 23:40:33.626041 28029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:33.611824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59294 (local address 127.18.186.131:42865)
0808 23:40:33.612030 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:33.612037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:33.612056 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:40:33.612260 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:33.612266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:33.612342 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:33.612461 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:33.613778 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.614501 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.615779 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:33.616177 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:33.616318 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:33.620564 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:33.620579 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:33.620594 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:33.620627 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:33.623278 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:33.623970 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:33.623977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:33.623979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:33.624099 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:33.624515 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:33.624519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:33.624521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:33.624900 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:33.625238 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:33.625477 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 23:40:33.625718 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:40:33.626704 27895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:33.754637 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 27750
I20260808 23:40:33.780464 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 27335
I20260808 23:40:33.781692 27413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:33.897969 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 27335
2026-08-08T23:40:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5733 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:40:34 dist-test-slave-5k54 krb5kdc[28046](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:40:34 dist-test-slave-5k54 krb5kdc[28046](info): set up 2 sockets
Aug 08 23:40:34 dist-test-slave-5k54 krb5kdc[28046](info): commencing operation
krb5kdc: starting...
W20260808 23:40:35.047847 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:40:35 dist-test-slave-5k54 krb5kdc[28046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:40:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:40:35.300933 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:34497
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:34497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:35.631157 28059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:35.631510 28059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:35.631675 28059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:35.638954 28059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:35.639068 28059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:35.639168 28059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:35.639276 28059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:35.639370 28059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:35.649302 28059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:34497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:34497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:35.650601 28059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:35.652047 28059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:35.664026 28065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:35.663962 28068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:35.663910 28066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:35.665223 28059 server_base.cc:1034] running on GCE node
I20260808 23:40:35.666080 28059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:35.667585 28059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:35.668804 28059 hybrid_clock.cc:648] HybridClock initialized: now 1786232435668753 us; error 36 us; skew 500 ppm
I20260808 23:40:35.669173 28059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T23:40:35Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1837 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:40:35 dist-test-slave-5k54 krb5kdc[28074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:40:35 dist-test-slave-5k54 krb5kdc[28074](info): set up 2 sockets
Aug 08 23:40:35 dist-test-slave-5k54 krb5kdc[28074](info): commencing operation
krb5kdc: starting...
W20260808 23:40:36.906189 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:40:37 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:40:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:40:37.156776 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:36651
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:36651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:37.485127 28087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:37.485412 28087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:37.485620 28087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:37.492913 28087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:37.493017 28087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:37.493134 28087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:37.493224 28087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:37.493304 28087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:37.503055 28087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:36651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:36651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:37.504405 28087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:37.505749 28087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:37.518224 28092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:37.518170 28095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:37.518203 28093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:37.519035 28087 server_base.cc:1034] running on GCE node
I20260808 23:40:37.520021 28087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:37.521492 28087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:37.522719 28087 hybrid_clock.cc:648] HybridClock initialized: now 1786232437522666 us; error 42 us; skew 500 ppm
I20260808 23:40:37.523063 28087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:37 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232437, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:37.529748 28087 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:40:37.531651 28087 webserver.cc:460] Webserver started at http://127.18.186.190:45407/ using document root <none> and password file <none>
I20260808 23:40:37.532322 28087 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:37.532428 28087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:37.532684 28087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:37.535650 28087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "a4a27a1d6e434e3c80ec98ce903f736f"
format_stamp: "Formatted at 2026-08-08 23:40:37 on dist-test-slave-5k54"
server_key: "bb769d3d4dd8c6a6937a99e7f9abfbd5"
server_key_iv: "8881347062ad323f6c172efc8138a83e"
server_key_version: "encryptionkey@0"
I20260808 23:40:37.536357 28087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "a4a27a1d6e434e3c80ec98ce903f736f"
format_stamp: "Formatted at 2026-08-08 23:40:37 on dist-test-slave-5k54"
server_key: "bb769d3d4dd8c6a6937a99e7f9abfbd5"
server_key_iv: "8881347062ad323f6c172efc8138a83e"
server_key_version: "encryptionkey@0"
I20260808 23:40:37.541376 28087 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:40:37.544746 28102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:37.546036 28087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:40:37.546223 28087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "a4a27a1d6e434e3c80ec98ce903f736f"
format_stamp: "Formatted at 2026-08-08 23:40:37 on dist-test-slave-5k54"
server_key: "bb769d3d4dd8c6a6937a99e7f9abfbd5"
server_key_iv: "8881347062ad323f6c172efc8138a83e"
server_key_version: "encryptionkey@0"
I20260808 23:40:37.546429 28087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:37.558841 28087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:37.562724 28087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:37.563045 28087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:37.582043 28087 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:36651
I20260808 23:40:37.582077 28159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:36651 every 8 connection(s)
I20260808 23:40:37.583468 28087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:37.587553 28160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:37.591339 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 28087
I20260808 23:40:37.591635 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:37.592566 19178 external_mini_cluster.cc:1428] Setting key 915cb71767f2ec8cb950b3cdd381d1ff
I20260808 23:40:37.601904 28160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Bootstrap starting.
I20260808 23:40:37.605999 28160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:37.607446 28160 log.cc:824] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:40:37 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:37.610558 28160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: No bootstrap required, opened a new log
I20260808 23:40:37.617519 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:37.594741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.18.186.190:36651)
0808 23:40:37.595559 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:37.595570 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:37.595612 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:40:37.597243 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:37.597265 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:37.597656 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:37.597977 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:37.600007 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:37.601272 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:37.603133 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:37.603885 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:37.604023 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:37.609537 (+ 5514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:37.609545 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:37.609560 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:37.609596 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:37.613090 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:37.613897 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:37.613914 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:37.613920 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:37.614120 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:37.614545 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:37.614552 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:37.614555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:37.615116 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:37.615524 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:37.615991 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 23:40:37.616456 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":367,"threads_started":1}
I20260808 23:40:37.619395 28160 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } }
I20260808 23:40:37.619868 28160 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:37.620013 28160 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4a27a1d6e434e3c80ec98ce903f736f, State: Initialized, Role: FOLLOWER
I20260808 23:40:37.620678 28160 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } }
I20260808 23:40:37.620940 28160 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:37.621101 28160 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:37.621274 28160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:37.623688 28160 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } }
I20260808 23:40:37.624261 28160 leader_election.cc:302] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4a27a1d6e434e3c80ec98ce903f736f; no voters:
I20260808 23:40:37.624878 28160 leader_election.cc:288] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:37.625044 28166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:37.626375 28166 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 1 LEADER]: Becoming Leader. State: Replica: a4a27a1d6e434e3c80ec98ce903f736f, State: Running, Role: LEADER
I20260808 23:40:37.626895 28166 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } }
I20260808 23:40:37.627251 28160 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:37.629607 28169 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4a27a1d6e434e3c80ec98ce903f736f. Latest consensus state: current_term: 1 leader_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } } }
I20260808 23:40:37.629969 28169 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:37.630576 28167 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } } }
I20260808 23:40:37.630863 28167 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:37.630918 28175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:37.637814 28175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:37.647789 28175 catalog_manager.cc:1347] Generated new cluster ID: 9643cad4ee7d4a5d98eb83252ba9dee8
I20260808 23:40:37.647915 28175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:37.663338 28175 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:37.665259 28175 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:37.675371 28175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Generated new TSK 0
I20260808 23:40:37.676371 28175 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:40:37.747208 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:36651
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:38.081346 28190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:38.081619 28190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:38.081715 28190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:38.088749 28190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:38.088855 28190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:38.088989 28190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:38.098426 28190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:36651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:38.099766 28190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:38.101023 28190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:38.115084 28196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:38.115135 28199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:38.115159 28197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:38.115777 28190 server_base.cc:1034] running on GCE node
I20260808 23:40:38.116453 28190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:38.117432 28190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:38.118664 28190 hybrid_clock.cc:648] HybridClock initialized: now 1786232438118620 us; error 36 us; skew 500 ppm
I20260808 23:40:38.118990 28190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:38 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:38.122246 28190 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:40:38.123843 28190 webserver.cc:460] Webserver started at http://127.18.186.129:41475/ using document root <none> and password file <none>
I20260808 23:40:38.124536 28190 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:38.124693 28190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:38.125021 28190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:38.128118 28190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "4848289860a64045a212973e9b2e1a47"
format_stamp: "Formatted at 2026-08-08 23:40:38 on dist-test-slave-5k54"
server_key: "b1362f163cf1de0412aaf355c6c1eefd"
server_key_iv: "80d161f1c952fad759545d69f8fd7b54"
server_key_version: "encryptionkey@0"
I20260808 23:40:38.128997 28190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "4848289860a64045a212973e9b2e1a47"
format_stamp: "Formatted at 2026-08-08 23:40:38 on dist-test-slave-5k54"
server_key: "b1362f163cf1de0412aaf355c6c1eefd"
server_key_iv: "80d161f1c952fad759545d69f8fd7b54"
server_key_version: "encryptionkey@0"
I20260808 23:40:38.134548 28190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:40:38.137939 28206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:38.139209 28190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:40:38.139384 28190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "4848289860a64045a212973e9b2e1a47"
format_stamp: "Formatted at 2026-08-08 23:40:38 on dist-test-slave-5k54"
server_key: "b1362f163cf1de0412aaf355c6c1eefd"
server_key_iv: "80d161f1c952fad759545d69f8fd7b54"
server_key_version: "encryptionkey@0"
I20260808 23:40:38.139638 28190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:38.158627 28190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:38.162503 28190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:38.162796 28190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:38.163859 28190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:38.165727 28190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:38.165802 28190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:38.165895 28190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:38.165939 28190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:38.199215 28190 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:36733
I20260808 23:40:38.199309 28329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:36733 every 8 connection(s)
I20260808 23:40:38.200634 28190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:38.202683 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 28190
I20260808 23:40:38.202912 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:40:38.203830 19178 external_mini_cluster.cc:1428] Setting key 9b1c053c16dbf42e3880d97fecebc4d7
Aug 08 23:40:38 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:38.224792 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:38.203419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:47633 (local address 127.18.186.190:36651)
0808 23:40:38.204008 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:38.204015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:38.205107 (+ 1092us) server_negotiation.cc:407] Connection header received
0808 23:40:38.207709 (+ 2602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:38.207713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:38.207778 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:38.207872 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:38.210156 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:38.210910 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:38.213464 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:38.214099 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:38.214230 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:38.218610 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:38.218616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:38.218619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:38.218652 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:38.221286 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:38.222315 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:38.222319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:38.222321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:38.222451 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:38.222939 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:38.222942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:38.222944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:38.223220 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:38.223360 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:38.224349 (+ 989us) server_negotiation.cc:299] Negotiation successful
0808 23:40:38.224536 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":258,"threads_started":1}
I20260808 23:40:38.226399 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:38.204095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:47633)
0808 23:40:38.204928 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:38.204978 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:38.207350 (+ 2372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:38.208186 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:38.208201 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:38.209058 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:38.209932 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:38.209964 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:38.211094 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:38.211102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:38.213293 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:38.213311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:38.214356 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:38.214366 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:38.214697 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:38.215533 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:38.215574 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:38.218379 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:38.221440 (+ 3061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:38.221453 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:38.221472 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:38.222193 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:38.222557 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:38.222565 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:38.222572 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:38.222818 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:38.223369 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:38.223408 (+ 39us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:38.223930 (+ 522us) client_negotiation.cc:769] Sending connection context
0808 23:40:38.224252 (+ 322us) client_negotiation.cc:240] Negotiation successful
0808 23:40:38.224864 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":316,"threads_started":1}
I20260808 23:40:38.228268 28330 heartbeater.cc:347] Connected to a master server at 127.18.186.190:36651
I20260808 23:40:38.228699 28330 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:38.229642 28330 heartbeater.cc:511] Master 127.18.186.190:36651 requested a full tablet report, sending...
I20260808 23:40:38.232014 28121 ts_manager.cc:194] Registered new tserver with Master: 4848289860a64045a212973e9b2e1a47 (127.18.186.129:36733)
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
I20260808 23:40:38.233626 28121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:47633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:40:38.264904 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:36651
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:38.598436 28336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:38.598703 28336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:38.598909 28336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:38.606117 28336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:38.606225 28336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:38.606413 28336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:38.616006 28336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:36651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:38.617242 28336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:38.618556 28336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:38.632582 28342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:38.632709 28343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:38.632711 28345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:38.633719 28336 server_base.cc:1034] running on GCE node
I20260808 23:40:38.634254 28336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:38.635152 28336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:38.636356 28336 hybrid_clock.cc:648] HybridClock initialized: now 1786232438636270 us; error 69 us; skew 500 ppm
I20260808 23:40:38.636653 28336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:38 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:38.639811 28336 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:40:38.641340 28336 webserver.cc:460] Webserver started at http://127.18.186.130:39903/ using document root <none> and password file <none>
I20260808 23:40:38.642030 28336 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:38.642175 28336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:38.642500 28336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:38.645430 28336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "09a13803c4da4c7ba84297fc28e5297e"
format_stamp: "Formatted at 2026-08-08 23:40:38 on dist-test-slave-5k54"
server_key: "a7404a57967371072938e124df7a11fa"
server_key_iv: "9c816a91a20d4f64c298e189e92e92b4"
server_key_version: "encryptionkey@0"
I20260808 23:40:38.646313 28336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "09a13803c4da4c7ba84297fc28e5297e"
format_stamp: "Formatted at 2026-08-08 23:40:38 on dist-test-slave-5k54"
server_key: "a7404a57967371072938e124df7a11fa"
server_key_iv: "9c816a91a20d4f64c298e189e92e92b4"
server_key_version: "encryptionkey@0"
I20260808 23:40:38.651569 28336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260808 23:40:38.654896 28353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:38.656297 28336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:38.656481 28336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "09a13803c4da4c7ba84297fc28e5297e"
format_stamp: "Formatted at 2026-08-08 23:40:38 on dist-test-slave-5k54"
server_key: "a7404a57967371072938e124df7a11fa"
server_key_iv: "9c816a91a20d4f64c298e189e92e92b4"
server_key_version: "encryptionkey@0"
I20260808 23:40:38.656687 28336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:38.705757 28336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:38.710254 28336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:38.710554 28336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:38.711575 28336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:38.712910 28336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:38.713013 28336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:38.713165 28336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:38.713238 28336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:38.747754 28336 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:44931
I20260808 23:40:38.747812 28478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:44931 every 8 connection(s)
I20260808 23:40:38.749058 28336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:38.751044 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 28336
I20260808 23:40:38.751235 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:40:38.751775 19178 external_mini_cluster.cc:1428] Setting key 8d6a607dbc595b2d0312cb0ef5503bd0
Aug 08 23:40:38 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:38.773075 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:38.752012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:59089 (local address 127.18.186.190:36651)
0808 23:40:38.752537 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:38.752546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:38.753736 (+ 1190us) server_negotiation.cc:407] Connection header received
0808 23:40:38.756562 (+ 2826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:38.756566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:38.756630 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:38.756722 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:38.758795 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:38.759628 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:38.762182 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:38.762621 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:38.762715 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:38.767107 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:38.767112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:38.767115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:38.767147 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:38.769773 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:38.770695 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:38.770699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:38.770701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:38.770829 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:38.771347 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:38.771350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:38.771352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:38.771662 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:38.771787 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:38.772645 (+ 858us) server_negotiation.cc:299] Negotiation successful
0808 23:40:38.772807 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":249,"threads_started":1}
I20260808 23:40:38.774744 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:38.752718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:59089)
0808 23:40:38.753581 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:38.753644 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:38.756254 (+ 2610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:38.757004 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:38.757017 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:38.757801 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:38.758614 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:38.758641 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:38.759827 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:38.759842 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:38.762027 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:38.762046 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:38.762847 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:38.762856 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:38.763184 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:38.763933 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:38.763967 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:38.766877 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:38.769914 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:38.769926 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:38.769944 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:38.770587 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:38.770942 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:38.770950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:38.770956 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:38.771238 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:38.771786 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:38.771796 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:38.772242 (+ 446us) client_negotiation.cc:769] Sending connection context
0808 23:40:38.772610 (+ 368us) client_negotiation.cc:240] Negotiation successful
0808 23:40:38.773191 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":322,"threads_started":1}
I20260808 23:40:38.776357 28479 heartbeater.cc:347] Connected to a master server at 127.18.186.190:36651
I20260808 23:40:38.776701 28479 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:38.777623 28479 heartbeater.cc:511] Master 127.18.186.190:36651 requested a full tablet report, sending...
I20260808 23:40:38.779379 28121 ts_manager.cc:194] Registered new tserver with Master: 09a13803c4da4c7ba84297fc28e5297e (127.18.186.130:44931)
I20260808 23:40:38.780652 28121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:59089
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:40:38.815840 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:36651
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:39.149190 28486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:39.149489 28486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:39.149624 28486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:39.157168 28486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:39.157271 28486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:39.157434 28486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:39.167088 28486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:36651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:39.168398 28486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:39.169770 28486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:39.183100 28492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:39.183169 28494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:39.183197 28491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:39.184309 28486 server_base.cc:1034] running on GCE node
I20260808 23:40:39.185103 28486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:39.186088 28486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:39.187364 28486 hybrid_clock.cc:648] HybridClock initialized: now 1786232439187299 us; error 44 us; skew 500 ppm
I20260808 23:40:39.187731 28486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:39 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:39.191192 28486 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:40:39.192839 28486 webserver.cc:460] Webserver started at http://127.18.186.131:33383/ using document root <none> and password file <none>
I20260808 23:40:39.193526 28486 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:39.193642 28486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:39.193928 28486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:39.197042 28486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "d237b1ffae8d4f3883f7427b6646c69c"
format_stamp: "Formatted at 2026-08-08 23:40:39 on dist-test-slave-5k54"
server_key: "b70dfa511271a4a0aead95d9320674b7"
server_key_iv: "94fdbdc423f7ca60484a0e7b323cb4e7"
server_key_version: "encryptionkey@0"
I20260808 23:40:39.197851 28486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "d237b1ffae8d4f3883f7427b6646c69c"
format_stamp: "Formatted at 2026-08-08 23:40:39 on dist-test-slave-5k54"
server_key: "b70dfa511271a4a0aead95d9320674b7"
server_key_iv: "94fdbdc423f7ca60484a0e7b323cb4e7"
server_key_version: "encryptionkey@0"
I20260808 23:40:39.203558 28486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 23:40:39.207080 28501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:39.208599 28486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:39.208757 28486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "d237b1ffae8d4f3883f7427b6646c69c"
format_stamp: "Formatted at 2026-08-08 23:40:39 on dist-test-slave-5k54"
server_key: "b70dfa511271a4a0aead95d9320674b7"
server_key_iv: "94fdbdc423f7ca60484a0e7b323cb4e7"
server_key_version: "encryptionkey@0"
I20260808 23:40:39.209005 28486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:39.226120 28486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:39.230734 28486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:39.231050 28486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:39.232199 28486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:39.233731 28486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:39.233815 28486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:39.233927 28486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:39.233975 28486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:39.237022 28330 heartbeater.cc:503] Master 127.18.186.190:36651 was elected leader, sending a full tablet report...
I20260808 23:40:39.270612 28486 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:44089
I20260808 23:40:39.270722 28622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:44089 every 8 connection(s)
I20260808 23:40:39.271960 28486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:39.274356 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 28486
I20260808 23:40:39.274542 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:40:39.275169 19178 external_mini_cluster.cc:1428] Setting key 9d27d07b385b8e8a8487bff3182c5e9d
Aug 08 23:40:39 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:40:39.293239 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:39.274951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:59469 (local address 127.18.186.190:36651)
0808 23:40:39.275467 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:39.275473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:39.276683 (+ 1210us) server_negotiation.cc:407] Connection header received
0808 23:40:39.279074 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:39.279079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:39.279184 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:39.279301 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:39.280891 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.281688 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.283319 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.283811 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.283969 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:39.287365 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:39.287370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:39.287373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:39.287405 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:39.290184 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:39.290901 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:39.290913 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:39.290917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:39.291036 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:39.291522 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:39.291527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:39.291536 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:39.291808 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:39.291975 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:39.292635 (+ 660us) server_negotiation.cc:299] Negotiation successful
0808 23:40:39.292873 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":305,"threads_started":1}
I20260808 23:40:39.294176 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:39.275637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:59469)
0808 23:40:39.276500 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:39.276559 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:39.278805 (+ 2246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:39.279586 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:39.279594 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:39.280120 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:39.280728 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:39.280744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.281852 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.281856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:39.283160 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:39.283172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.284059 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.284069 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:39.284320 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:39.285021 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:39.285043 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:39.287161 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:39.290344 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:39.290357 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:39.290368 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:39.290772 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:39.291146 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:40:39.291152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:40:39.291155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:40:39.291354 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:40:39.291935 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:39.291943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:39.292266 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 23:40:39.292531 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 23:40:39.293024 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":328,"threads_started":1}
I20260808 23:40:39.295580 28623 heartbeater.cc:347] Connected to a master server at 127.18.186.190:36651
I20260808 23:40:39.295867 28623 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:39.296523 28623 heartbeater.cc:511] Master 127.18.186.190:36651 requested a full tablet report, sending...
I20260808 23:40:39.297853 28121 ts_manager.cc:194] Registered new tserver with Master: d237b1ffae8d4f3883f7427b6646c69c (127.18.186.131:44089)
I20260808 23:40:39.298664 28121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:59469
I20260808 23:40:39.302882 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab.
Entry for principal oryx/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab.
WARNING: no policy specified for oryx/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab.
Entry for principal oryx/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab.
WARNING: no policy specified for oryx/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab.
Entry for principal oryx/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab.
I20260808 23:40:39.464061 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28190
Aug 08 23:40:39 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:40:39.480932 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:39.465427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54240 (local address 127.18.186.129:36733)
0808 23:40:39.466075 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:39.466082 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:39.466109 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:40:39.466173 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:39.466179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:39.466405 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:39.466557 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:39.468021 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.469045 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.470311 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.470868 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.470965 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:39.474952 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:39.474963 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:39.474977 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:39.475010 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:39.477942 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:39.478651 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:39.478658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:39.478660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:39.478790 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:39.479182 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:39.479188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:39.479191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:39.479599 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:39.479920 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:39.480219 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 23:40:39.480506 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":339,"threads_started":1}
I20260808 23:40:39.482169 28325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:39.565833 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 28190
I20260808 23:40:39.586081 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28336
Aug 08 23:40:39 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:40:39.603578 28636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:39.587044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.18.186.130:44931)
0808 23:40:39.587700 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:39.587709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:39.587739 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:40:39.587809 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:39.587817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:39.588114 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:39.588275 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:39.589472 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.591089 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.592456 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.593223 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.593320 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:39.597560 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:39.597572 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:39.597585 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:39.597617 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:39.601123 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:39.601754 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:39.601760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:39.601762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:39.601884 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:39.602202 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:39.602205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:39.602207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:39.602520 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:39.602739 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:39.603007 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 23:40:39.603199 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":382,"threads_started":1}
I20260808 23:40:39.604875 28474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:39.688335 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 28336
I20260808 23:40:39.706619 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28486
Aug 08 23:40:39 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:40:39.723773 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:39.707931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.18.186.131:44089)
0808 23:40:39.708406 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:39.708413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:39.708440 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:40:39.708506 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:39.708513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:39.708713 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:39.708840 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:39.710090 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.711046 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.712347 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:39.713000 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:39.713096 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:39.717760 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:39.717770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:39.717783 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:39.717815 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:39.720973 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:39.721662 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:39.721670 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:39.721672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:39.721798 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:39.722194 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:39.722198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:39.722200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:39.722534 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:39.722826 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:39.723144 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 23:40:39.723415 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":221,"threads_started":1}
I20260808 23:40:39.725062 28618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:39.807757 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 28486
I20260808 23:40:39.826308 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28087
I20260808 23:40:39.827570 28154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:39.931273 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 28087
I20260808 23:40:39.954638 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:36651
--webserver_interface=127.18.186.190
--webserver_port=45407
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:36651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:40:40.281490 28641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:40.281853 28641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:40.281956 28641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:40.289034 28641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:40.289124 28641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:40.289171 28641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:40.289227 28641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:40.289270 28641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:40.298473 28641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:36651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:36651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=45407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:40.299680 28641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:40.300870 28641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:40.315655 28647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:40.315742 28649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:40.315760 28646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:40.316596 28641 server_base.cc:1034] running on GCE node
I20260808 23:40:40.317351 28641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:40.318784 28641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:40.320008 28641 hybrid_clock.cc:648] HybridClock initialized: now 1786232440319887 us; error 138 us; skew 500 ppm
I20260808 23:40:40.320340 28641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:40 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:40.323390 28641 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:40:40.324982 28641 webserver.cc:460] Webserver started at http://127.18.186.190:45407/ using document root <none> and password file <none>
I20260808 23:40:40.325664 28641 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:40.325822 28641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:40.331123 28641 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:40.333979 28657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:40.335289 28641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:40:40.335544 28641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "a4a27a1d6e434e3c80ec98ce903f736f"
format_stamp: "Formatted at 2026-08-08 23:40:37 on dist-test-slave-5k54"
server_key: "bb769d3d4dd8c6a6937a99e7f9abfbd5"
server_key_iv: "8881347062ad323f6c172efc8138a83e"
server_key_version: "encryptionkey@0"
I20260808 23:40:40.336491 28641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:40.349603 28641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:40.353425 28641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:40.353703 28641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:40.372630 28641 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:36651
I20260808 23:40:40.372651 28715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:36651 every 8 connection(s)
I20260808 23:40:40.374097 28641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:40.378050 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 28641
I20260808 23:40:40.379088 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:36733
--local_ip_for_outbound_sockets=127.18.186.129
--tserver_master_addrs=127.18.186.190:36651
--webserver_port=41475
--webserver_interface=127.18.186.129
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.186.129
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:40:40.382530 28716 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:40:40.387748 28716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Bootstrap starting.
I20260808 23:40:40.407169 28716 log.cc:824] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:40.416028 28716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:40:40.416561 28716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Bootstrap complete.
I20260808 23:40:40.426664 28716 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } }
I20260808 23:40:40.427644 28716 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4a27a1d6e434e3c80ec98ce903f736f, State: Initialized, Role: FOLLOWER
I20260808 23:40:40.428449 28716 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } }
I20260808 23:40:40.428785 28716 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:40.428951 28716 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:40.429188 28716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:40:40.433359 28716 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } }
I20260808 23:40:40.433903 28716 leader_election.cc:302] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4a27a1d6e434e3c80ec98ce903f736f; no voters:
I20260808 23:40:40.434629 28716 leader_election.cc:288] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:40:40.434775 28720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:40:40.436132 28720 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [term 2 LEADER]: Becoming Leader. State: Replica: a4a27a1d6e434e3c80ec98ce903f736f, State: Running, Role: LEADER
I20260808 23:40:40.436839 28720 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } }
I20260808 23:40:40.437050 28716 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:40.439455 28722 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4a27a1d6e434e3c80ec98ce903f736f. Latest consensus state: current_term: 2 leader_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } } }
I20260808 23:40:40.439833 28722 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:40.440585 28721 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a27a1d6e434e3c80ec98ce903f736f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 36651 } } }
I20260808 23:40:40.441264 28721 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:40.441915 28728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:40.462219 28728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:40.463796 28728 catalog_manager.cc:1259] Loaded cluster ID: 9643cad4ee7d4a5d98eb83252ba9dee8
I20260808 23:40:40.463960 28728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:40.468353 28728 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:40.469978 28728 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4a27a1d6e434e3c80ec98ce903f736f: Loaded TSK: 0
I20260808 23:40:40.471793 28728 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:40:40.727898 28718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:40.728180 28718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:40.728331 28718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:40.735625 28718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:40.735723 28718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:40.735885 28718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:40.746004 28718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.129.keytab
--principal=oryx/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:36733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=41475
--tserver_master_addrs=127.18.186.190:36651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:40.747293 28718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:40.748631 28718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:40.763289 28744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:40.763321 28746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:40.763948 28748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:40.764508 28718 server_base.cc:1034] running on GCE node
I20260808 23:40:40.765113 28718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:40.766129 28718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:40.767376 28718 hybrid_clock.cc:648] HybridClock initialized: now 1786232440767321 us; error 40 us; skew 500 ppm
I20260808 23:40:40.767742 28718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:40 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232440, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:40.771004 28718 init.cc:377] Logged in from keytab as oryx/127.18.186.129@KRBTEST.COM (short username oryx)
I20260808 23:40:40.772531 28718 webserver.cc:460] Webserver started at http://127.18.186.129:41475/ using document root <none> and password file <none>
I20260808 23:40:40.773226 28718 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:40.773384 28718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:40.778669 28718 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:40:40.781502 28755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:40.782919 28718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 23:40:40.783115 28718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "4848289860a64045a212973e9b2e1a47"
format_stamp: "Formatted at 2026-08-08 23:40:38 on dist-test-slave-5k54"
server_key: "b1362f163cf1de0412aaf355c6c1eefd"
server_key_iv: "80d161f1c952fad759545d69f8fd7b54"
server_key_version: "encryptionkey@0"
I20260808 23:40:40.784179 28718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:40.799072 28718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:40.803170 28718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:40.803519 28718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:40.804509 28718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:40.805977 28718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:40.806103 28718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:40.806231 28718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:40.806322 28718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:40.840329 28718 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:36733
I20260808 23:40:40.840385 28879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:36733 every 8 connection(s)
I20260808 23:40:40.841795 28718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:40.847708 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 28718
I20260808 23:40:40.848464 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:44931
--local_ip_for_outbound_sockets=127.18.186.130
--tserver_master_addrs=127.18.186.190:36651
--webserver_port=39903
--webserver_interface=127.18.186.130
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.186.130
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:40:40 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:40 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:40.862149 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:40.844890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:53555)
0808 23:40:40.845700 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:40.845743 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:40.847819 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:40.849307 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:40.849321 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:40.850132 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:40.850957 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:40.850984 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.852722 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.852730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:40.854875 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:40.854893 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.855968 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.855978 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:40.856285 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:40.857117 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:40.857150 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:40.861215 (+ 4065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":346,"threads_started":1}
W20260808 23:40:40.862833 28880 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:36651 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.186.190:36651: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
W20260808 23:40:40.862866 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:40.844881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:53555 (local address 127.18.186.190:36651)
0808 23:40:40.845552 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:40.845563 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:40.845860 (+ 297us) server_negotiation.cc:407] Connection header received
0808 23:40:40.848376 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:40.848389 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:40.848772 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:40.849073 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:40.851153 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.852564 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.855033 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.855739 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.855872 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:40.861656 (+ 5784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:53555: BlockingRecv error: recv got EOF from 127.18.186.129:53555 (error 108)
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":257,"threads_started":1}
Aug 08 23:40:40 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:40 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:40.872314 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:40.863535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:48079)
0808 23:40:40.863720 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:40.863745 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:40.863863 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:40.864229 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:40.864233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:40.864572 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:40.864705 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:40.864712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.865752 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.865758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:40.866641 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:40.866652 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.867327 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.867332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:40.867537 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:40.868139 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:40.868156 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:40.871955 (+ 3799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 23:40:40.873283 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:40.863757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:48079 (local address 127.18.186.190:36651)
0808 23:40:40.863968 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:40.863974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:40.863990 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:40:40.864045 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:40.864048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:40.864101 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:40.864203 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:40.864842 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.865612 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.866780 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.867187 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.867316 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:40.873056 (+ 5740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:48079: BlockingRecv error: recv got EOF from 127.18.186.129:48079 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 23:40:40 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:40 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:40.883198 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:40.873248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:36797)
0808 23:40:40.873899 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:40.873927 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:40.874084 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:40.875328 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:40.875333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:40.875788 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:40.875986 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:40.876001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.877620 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.877624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:40.878553 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:40.878564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.879504 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.879510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:40.879670 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:40.880247 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:40.880263 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:40.882982 (+ 2719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":1501,"threads_started":1}
W20260808 23:40:40.883489 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:40.874347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:36797 (local address 127.18.186.190:36651)
0808 23:40:40.874605 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:40.874612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:40.874634 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:40:40.875096 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:40.875103 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:40.875190 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:40.875557 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:40.876208 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.877471 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.878694 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:40.879333 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:40.879477 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:40.883213 (+ 3736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:36797: BlockingRecv error: recv got EOF from 127.18.186.129:36797 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
W20260808 23:40:40.883767 28880 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:40:41.178129 28885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:41.178393 28885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:41.178572 28885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:41.186000 28885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:41.186095 28885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:41.186272 28885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:41.195796 28885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.130.keytab
--principal=oryx/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:44931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=39903
--tserver_master_addrs=127.18.186.190:36651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:41.197093 28885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:41.198364 28885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:41.211212 28891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:41.211181 28892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:41.211155 28895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:41.212070 28885 server_base.cc:1034] running on GCE node
I20260808 23:40:41.212610 28885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:41.213479 28885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:41.214669 28885 hybrid_clock.cc:648] HybridClock initialized: now 1786232441214617 us; error 55 us; skew 500 ppm
I20260808 23:40:41.214943 28885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232441, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:41.218204 28885 init.cc:377] Logged in from keytab as oryx/127.18.186.130@KRBTEST.COM (short username oryx)
I20260808 23:40:41.219800 28885 webserver.cc:460] Webserver started at http://127.18.186.130:39903/ using document root <none> and password file <none>
I20260808 23:40:41.220495 28885 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:41.220647 28885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:41.226013 28885 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:40:41.228647 28904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:41.229979 28885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:41.230159 28885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "09a13803c4da4c7ba84297fc28e5297e"
format_stamp: "Formatted at 2026-08-08 23:40:38 on dist-test-slave-5k54"
server_key: "a7404a57967371072938e124df7a11fa"
server_key_iv: "9c816a91a20d4f64c298e189e92e92b4"
server_key_version: "encryptionkey@0"
I20260808 23:40:41.231099 28885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:41.245443 28885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:41.249589 28885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:41.249866 28885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:41.250819 28885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:41.252153 28885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:41.252270 28885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:41.252409 28885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:41.252498 28885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:41.285943 28885 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:44931
I20260808 23:40:41.286031 29026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:44931 every 8 connection(s)
I20260808 23:40:41.287592 28885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:41.296164 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 28885
I20260808 23:40:41.296779 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:44089
--local_ip_for_outbound_sockets=127.18.186.131
--tserver_master_addrs=127.18.186.190:36651
--webserver_port=33383
--webserver_interface=127.18.186.131
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.186.131
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:40:41.303809 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:41.290129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:38207 (local address 127.18.186.190:36651)
0808 23:40:41.290311 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:41.290316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:41.291489 (+ 1173us) server_negotiation.cc:407] Connection header received
0808 23:40:41.293310 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:41.293314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:41.293379 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:41.293473 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:41.294977 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.295848 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.297609 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.298395 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.298552 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.303510 (+ 4958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:38207: BlockingRecv error: recv got EOF from 127.18.186.130:38207 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:40:41.304316 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:41.290612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:38207)
0808 23:40:41.291304 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:41.291340 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:41.293066 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:41.293680 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:41.293688 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:41.294174 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:41.294724 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.294742 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.296027 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.296031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.297460 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.297472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.298606 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.298611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.298790 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.299556 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:41.299578 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:41.303222 (+ 3644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":257,"threads_started":1}
W20260808 23:40:41.305017 29027 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:36651 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.186.190:36651: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:41.313918 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:41.305563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:45405)
0808 23:40:41.305788 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:41.305808 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:41.305919 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:41.306208 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:41.306216 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:41.306662 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:41.306861 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.306869 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.308350 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.308356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.309287 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.309298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.310313 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.310317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.310438 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.311008 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:41.311024 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:41.313734 (+ 2710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 23:40:41.314970 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:41.305784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:45405 (local address 127.18.186.190:36651)
0808 23:40:41.305956 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:41.305960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:41.305975 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:40:41.306018 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:41.306022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:41.306076 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:41.306152 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:41.307008 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.308195 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.309438 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.310178 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.310316 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.314656 (+ 4340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:45405: BlockingRecv error: recv got EOF from 127.18.186.130:45405 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:41.325778 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:41.314628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:50533)
0808 23:40:41.314893 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:41.314914 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:41.315007 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:41.315701 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:41.315708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:41.316135 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:41.316370 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.316396 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.317326 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.317334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.319063 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.319080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.320183 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.320193 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.320547 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.321328 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:41.321357 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:41.325413 (+ 4056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140}
W20260808 23:40:41.326118 29027 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:40:41.326242 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:41.314792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:50533 (local address 127.18.186.190:36651)
0808 23:40:41.315400 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:41.315404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:41.315459 (+ 55us) server_negotiation.cc:407] Connection header received
0808 23:40:41.315512 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:41.315515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:41.315565 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:41.315640 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:41.316521 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.317187 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.319250 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.319995 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.320132 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.325655 (+ 5523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:50533: BlockingRecv error: recv got EOF from 127.18.186.130:50533 (error 108)
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":487,"threads_started":1}
W20260808 23:40:41.655623 29030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:41.655910 29030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:41.656005 29030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:41.663103 29030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:41.663223 29030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:41.663337 29030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:41.673247 29030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/krb5kdc/oryx_127.18.186.131.keytab
--principal=oryx/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:44089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=33383
--tserver_master_addrs=127.18.186.190:36651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:41.674588 29030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:41.675906 29030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:41.689267 29039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:41.689330 29037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:41.689361 29036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:41.690196 29030 server_base.cc:1034] running on GCE node
I20260808 23:40:41.690763 29030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:41.691912 29030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:41.693156 29030 hybrid_clock.cc:648] HybridClock initialized: now 1786232441693092 us; error 63 us; skew 500 ppm
I20260808 23:40:41.693511 29030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232441, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:40:41.696936 29030 init.cc:377] Logged in from keytab as oryx/127.18.186.131@KRBTEST.COM (short username oryx)
I20260808 23:40:41.698467 29030 webserver.cc:460] Webserver started at http://127.18.186.131:33383/ using document root <none> and password file <none>
I20260808 23:40:41.699174 29030 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:41.699344 29030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:41.706375 29030 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:40:41.709381 29049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:41.710785 29030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:40:41.710971 29030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "d237b1ffae8d4f3883f7427b6646c69c"
format_stamp: "Formatted at 2026-08-08 23:40:39 on dist-test-slave-5k54"
server_key: "b70dfa511271a4a0aead95d9320674b7"
server_key_iv: "94fdbdc423f7ca60484a0e7b323cb4e7"
server_key_version: "encryptionkey@0"
I20260808 23:40:41.712046 29030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:41.741510 29030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:41.746032 29030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:41.746330 29030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:41.747335 29030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:41.748759 29030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:41.748885 29030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:41.749029 29030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:41.749119 29030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:41.784083 29030 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:44089
I20260808 23:40:41.784175 29177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:44089 every 8 connection(s)
I20260808 23:40:41.785722 29030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:41.786770 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 29030
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:40:41.803594 29031 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:41.788771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:43699 (local address 127.18.186.190:36651)
0808 23:40:41.789947 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:41.789953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:41.790447 (+ 494us) server_negotiation.cc:407] Connection header received
0808 23:40:41.792927 (+ 2480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:41.792932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:41.793014 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:41.793174 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:41.795211 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.796057 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.797702 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.798402 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.798545 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.803295 (+ 4750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:43699: BlockingRecv error: recv got EOF from 127.18.186.131:43699 (error 108)
Metrics: {"server-negotiator.queue_time_us":1008}
I20260808 23:40:41.804183 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:41.789452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:43699)
0808 23:40:41.790272 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:41.790311 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:41.792656 (+ 2345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:41.793441 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:41.793450 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:41.794184 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:41.794974 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.794999 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.796224 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.796228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.797521 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.797533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.798619 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.798624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.798827 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.799485 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:41.799516 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:41.803008 (+ 3492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":392,"threads_started":1}
W20260808 23:40:41.805022 29178 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:36651 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.186.190:36651: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
I20260808 23:40:41.805277 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:41.788043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46586 (local address 127.18.186.190:36651)
0808 23:40:41.788328 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:41.788337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:41.788816 (+ 479us) server_negotiation.cc:407] Connection header received
0808 23:40:41.789026 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:41.789030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:41.789095 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:41.789181 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:41.791157 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.792376 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.793669 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.794292 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.794420 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.798007 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:41.798020 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:41.798043 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:41.798111 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:41.801546 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:41.802318 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:41.802325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:41.802332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:41.802502 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:40:41.802902 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:40:41.802908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:40:41.802912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:40:41.803589 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:41.803990 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:41.804484 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 23:40:41.804896 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:41.813750 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:41.805647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:52821)
0808 23:40:41.805824 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:41.805841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:41.805933 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:41.806251 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:41.806256 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:41.806651 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:41.806851 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.806863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.808125 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.808130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.809375 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.809394 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.810026 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.810030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.810147 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.810644 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:41.810660 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:41.813461 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 23:40:41.813903 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:41.805712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:52821 (local address 127.18.186.190:36651)
0808 23:40:41.805874 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:41.805879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:41.805895 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:40:41.806017 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:41.806021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:41.806110 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:41.806207 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:41.806998 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.807974 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.809503 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.809910 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.810002 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.813667 (+ 3665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:52821: BlockingRecv error: recv got EOF from 127.18.186.131:52821 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:41.822176 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:41.814602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:34131)
0808 23:40:41.814798 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:41.814819 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:41.814931 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:41.815195 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:41.815198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:41.815527 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:41.815674 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.815682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.816803 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.816806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.817773 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.817784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.818420 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.818424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.818551 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.819055 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:41.819071 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:41.821969 (+ 2898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 23:40:41.822367 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:41.814707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:34131 (local address 127.18.186.190:36651)
0808 23:40:41.814963 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:41.814967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:41.814983 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:40:41.815024 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:41.815028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:41.815078 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:41.815160 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:41.815835 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.816680 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.817898 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.818299 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.818394 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.822153 (+ 3759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:34131: BlockingRecv error: recv got EOF from 127.18.186.131:34131 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
W20260808 23:40:41.822541 29178 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:41 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:41.893096 29182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:41.884363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:49859)
0808 23:40:41.884868 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:41.884900 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:41.885060 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:41.885467 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:41.885471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:41.886008 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:41.886213 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.886222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.887264 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.887268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.888478 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:41.888490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.889173 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.889177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:41.889317 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.889900 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:41.889916 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:41.892875 (+ 2959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":248,"threads_started":1}
W20260808 23:40:41.893276 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:41.884426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:49859 (local address 127.18.186.190:36651)
0808 23:40:41.884647 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:41.884655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:41.884990 (+ 335us) server_negotiation.cc:407] Connection header received
0808 23:40:41.885195 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:41.885200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:41.885285 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:41.885378 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:41.886365 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.887122 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.888586 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:41.889016 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:41.889110 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:41.893042 (+ 3932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:49859: BlockingRecv error: recv got EOF from 127.18.186.129:49859 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 23:40:42 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:42 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:42.335899 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:42.326695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:59139)
0808 23:40:42.327266 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:42.327302 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:42.327503 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:42.327823 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:42.327826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:42.328306 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:42.328503 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:42.328512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.329456 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.329460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:42.330440 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:42.330454 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.331517 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.331525 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:42.331702 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:42.332407 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:42.332425 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:42.335660 (+ 3235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":329,"threads_started":1}
W20260808 23:40:42.336166 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:42.326765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:59139 (local address 127.18.186.190:36651)
0808 23:40:42.326940 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:42.326945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:42.327380 (+ 435us) server_negotiation.cc:407] Connection header received
0808 23:40:42.327627 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:42.327631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:42.327694 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:42.327776 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:42.328634 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.329327 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.330603 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.331248 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.331412 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:42.335871 (+ 4459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:59139: BlockingRecv error: recv got EOF from 127.18.186.130:59139 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 23:40:42 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:42 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:42.837415 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:42.823139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:43335)
0808 23:40:42.823849 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:42.823866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:42.823989 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:42.824289 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:42.824292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:42.824788 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:42.825024 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:42.825035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.826183 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.826187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:42.827164 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:42.827176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.828544 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.828550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:42.828697 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:42.833811 (+ 5114us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:42.833833 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:42.837017 (+ 3184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":419,"threads_started":1}
W20260808 23:40:42.837623 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:42.823216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:43335 (local address 127.18.186.190:36651)
0808 23:40:42.823399 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:42.823404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:42.823947 (+ 543us) server_negotiation.cc:407] Connection header received
0808 23:40:42.824087 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:42.824090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:42.824153 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:42.824236 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:42.825160 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.826018 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.827322 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.828105 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.828252 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:42.837357 (+ 9105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:43335: BlockingRecv error: recv got EOF from 127.18.186.131:43335 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 23:40:42 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:42 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:42.902907 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:42.893960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:36841)
0808 23:40:42.894539 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:42.894555 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:42.894707 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:42.895067 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:42.895072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:42.895645 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:42.895873 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:42.895911 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.897078 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.897085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:42.898240 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:42.898274 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.899058 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.899064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:42.899230 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:42.899823 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:42.899841 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:42.902547 (+ 2706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":317,"threads_started":1}
W20260808 23:40:42.903002 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:42.894146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:36841 (local address 127.18.186.190:36651)
0808 23:40:42.894378 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:42.894419 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:42.894636 (+ 217us) server_negotiation.cc:407] Connection header received
0808 23:40:42.894834 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:42.894840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:42.894928 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:42.895034 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:42.896037 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.896947 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.898390 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:42.898899 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:42.899046 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:42.902734 (+ 3688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:36841: BlockingRecv error: recv got EOF from 127.18.186.129:36841 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 23:40:43 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:43 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:43.345831 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:43.336746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:36381)
0808 23:40:43.337306 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:43.337322 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:43.337498 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:43.337921 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:43.337926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:43.338486 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:43.338716 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:43.338729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.340007 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.340011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:43.341059 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:43.341070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.341804 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.341808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:43.341921 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:43.342516 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:43.342533 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:43.345479 (+ 2946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":316,"threads_started":1}
W20260808 23:40:43.345918 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:43.336859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:36381 (local address 127.18.186.190:36651)
0808 23:40:43.337070 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:43.337075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:43.337397 (+ 322us) server_negotiation.cc:407] Connection header received
0808 23:40:43.337622 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:43.337625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:43.337714 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:43.337832 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:43.338877 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.339877 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.341225 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.341679 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.341771 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:43.345688 (+ 3917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:36381: BlockingRecv error: recv got EOF from 127.18.186.130:36381 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 23:40:43 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:43 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:43.847273 29188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:43.838822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:40205)
0808 23:40:43.839388 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:43.839414 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:43.839595 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:43.839927 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:43.839932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:43.840354 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:43.840527 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:43.840536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.841561 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.841568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:43.842517 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:43.842530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.843316 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.843322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:43.843540 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:43.844095 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:43.844113 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:43.846941 (+ 2828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":282,"threads_started":1}
W20260808 23:40:43.847462 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:43.838962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:40205 (local address 127.18.186.190:36651)
0808 23:40:43.839189 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:43.839197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:43.839515 (+ 318us) server_negotiation.cc:407] Connection header received
0808 23:40:43.839720 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:43.839726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:43.839796 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:43.839898 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:43.840684 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.841414 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.842654 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.843158 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.843294 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:43.847169 (+ 3875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:40205: BlockingRecv error: recv got EOF from 127.18.186.131:40205 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 23:40:43 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:43 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:43.912276 29189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:43.903905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:33775)
0808 23:40:43.904394 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:43.904420 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:43.904586 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:43.904931 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:43.904935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:43.905317 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:43.905490 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:43.905498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.906559 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.906566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:43.907506 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:43.907518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.908378 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.908383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:43.908561 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:43.909188 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:43.909223 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:43.911922 (+ 2699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":271,"threads_started":1}
W20260808 23:40:43.912374 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:43.903986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:33775 (local address 127.18.186.190:36651)
0808 23:40:43.904226 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:43.904234 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:43.904517 (+ 283us) server_negotiation.cc:407] Connection header received
0808 23:40:43.904710 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:43.904721 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:43.904793 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:43.904899 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:43.905641 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.906424 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.907650 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:43.908239 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:43.908370 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:43.912133 (+ 3763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:33775: BlockingRecv error: recv got EOF from 127.18.186.129:33775 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 08 23:40:44 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:44 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:44.355191 29190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:44.346782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:48557)
0808 23:40:44.347318 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:44.347335 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:44.347465 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:44.347802 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:44.347807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:44.348201 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:44.348363 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:44.348372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.349448 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.349454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:44.350351 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:44.350363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.351055 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.351061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:44.351242 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:44.351943 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:44.351961 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:44.354796 (+ 2835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":300,"threads_started":1}
W20260808 23:40:44.355312 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:44.346843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:48557 (local address 127.18.186.190:36651)
0808 23:40:44.347057 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:44.347064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:44.347416 (+ 352us) server_negotiation.cc:407] Connection header received
0808 23:40:44.347572 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:44.347576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:44.347654 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:44.347768 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:44.348524 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.349307 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.350509 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.350914 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.351041 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:44.355027 (+ 3986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:48557: BlockingRecv error: recv got EOF from 127.18.186.130:48557 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 23:40:44 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:44 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:44.856369 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:44.848359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:39377)
0808 23:40:44.848901 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:44.848919 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:44.849032 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:44.849318 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:44.849321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:44.849695 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:44.849856 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:44.849865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.850828 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.850832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:44.851781 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:44.851794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.852456 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.852460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:44.852581 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:44.853147 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:44.853164 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:44.856051 (+ 2887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":290,"threads_started":1}
W20260808 23:40:44.856570 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:44.848443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:39377 (local address 127.18.186.190:36651)
0808 23:40:44.848684 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:44.848694 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:44.849005 (+ 311us) server_negotiation.cc:407] Connection header received
0808 23:40:44.849131 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:44.849135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:44.849197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:44.849275 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:44.849985 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.850713 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.851911 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.852320 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.852413 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:44.856330 (+ 3917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:39377: BlockingRecv error: recv got EOF from 127.18.186.131:39377 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 23:40:44 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:44 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:44.922161 29192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:44.913346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:50447)
0808 23:40:44.913875 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:44.913890 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:44.914050 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:44.914380 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:44.914384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:44.914808 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:44.914964 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:44.914973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.916316 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.916321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:44.917441 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:44.917453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.918149 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.918155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:44.918329 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:44.918982 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:44.919000 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:44.921829 (+ 2829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":276,"threads_started":1}
W20260808 23:40:44.922343 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:44.913406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:50447 (local address 127.18.186.190:36651)
0808 23:40:44.913628 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:44.913635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:44.913976 (+ 341us) server_negotiation.cc:407] Connection header received
0808 23:40:44.914178 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:44.914182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:44.914245 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:44.914348 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:44.915136 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.916182 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.917582 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:44.918003 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:44.918136 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:44.922034 (+ 3898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:50447: BlockingRecv error: recv got EOF from 127.18.186.129:50447 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 23:40:45 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:45 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:45.364682 29193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:45.356185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:36501)
0808 23:40:45.356731 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:45.356747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:45.356948 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:45.357387 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:45.357392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:45.357916 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:45.358136 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:45.358152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.359120 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.359136 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:45.360090 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:45.360101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.360685 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.360689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:45.360797 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:45.361368 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:45.361386 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:45.364352 (+ 2966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":309,"threads_started":1}
W20260808 23:40:45.364823 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:45.356294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:36501 (local address 127.18.186.190:36651)
0808 23:40:45.356500 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:45.356506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:45.356870 (+ 364us) server_negotiation.cc:407] Connection header received
0808 23:40:45.357107 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:45.357112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:45.357216 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:45.357321 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:45.358279 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.359015 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.360202 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.360575 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.360662 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:45.364562 (+ 3900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:36501: BlockingRecv error: recv got EOF from 127.18.186.130:36501 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 23:40:45 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:45 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:45.865577 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:45.857643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:35085)
0808 23:40:45.858127 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:45.858144 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:45.858258 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:45.858579 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:45.858582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:45.858961 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:45.859106 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:45.859114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.860050 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.860054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:45.860995 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:45.861006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.861643 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.861647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:45.861764 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:45.862321 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:45.862338 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:45.865348 (+ 3010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":280,"threads_started":1}
W20260808 23:40:45.865782 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:45.857680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:35085 (local address 127.18.186.190:36651)
0808 23:40:45.857909 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:45.857915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:45.858229 (+ 314us) server_negotiation.cc:407] Connection header received
0808 23:40:45.858357 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:45.858361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:45.858437 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:45.858519 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:45.859228 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.859935 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.861108 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.861522 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.861606 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:45.865546 (+ 3940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:35085: BlockingRecv error: recv got EOF from 127.18.186.131:35085 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 23:40:45 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:45 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:45.931115 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:45.923163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:37067)
0808 23:40:45.923768 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:45.923784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:45.923903 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:45.924162 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:45.924165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:45.924567 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:45.924721 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:45.924729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.925736 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.925741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:45.926688 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:45.926701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.927373 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.927378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:45.927545 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:45.928116 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:45.928134 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:45.930875 (+ 2741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":340,"threads_started":1}
W20260808 23:40:45.931237 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:45.923234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:37067 (local address 127.18.186.190:36651)
0808 23:40:45.923398 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:45.923414 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:45.923855 (+ 441us) server_negotiation.cc:407] Connection header received
0808 23:40:45.923983 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:45.923986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:45.924043 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:45.924120 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:45.924873 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.925582 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.926814 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:45.927248 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:45.927340 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:45.931047 (+ 3707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:37067: BlockingRecv error: recv got EOF from 127.18.186.129:37067 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 23:40:46 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:46 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:46.377380 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:46.365698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:35243)
0808 23:40:46.366325 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:46.366341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:46.366452 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:46.366759 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:46.366763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:46.367174 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:46.367413 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:46.367429 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.368507 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.368511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:46.369433 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:46.369443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.370072 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.370076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:46.370188 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:46.374033 (+ 3845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:46.374050 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:46.377144 (+ 3094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":341,"threads_started":1}
W20260808 23:40:46.377665 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:46.365919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:35243 (local address 127.18.186.190:36651)
0808 23:40:46.366158 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:46.366164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:46.366423 (+ 259us) server_negotiation.cc:407] Connection header received
0808 23:40:46.366571 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:46.366575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:46.366637 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:46.366718 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:46.367607 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.368359 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.369552 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.369959 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.370052 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:46.377368 (+ 7316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:35243: BlockingRecv error: recv got EOF from 127.18.186.130:35243 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Aug 08 23:40:46 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:46 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:40:46.877825 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:46.866765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:39835 (local address 127.18.186.190:36651)
0808 23:40:46.866976 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:46.866984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:46.867540 (+ 556us) server_negotiation.cc:407] Connection header received
0808 23:40:46.867663 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:46.867666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:46.867731 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:46.867821 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:46.868542 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.869385 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.871726 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.872601 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.872748 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:46.877535 (+ 4787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:39835: BlockingRecv error: recv got EOF from 127.18.186.131:39835 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:40:46.877995 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:46.866695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:39835)
0808 23:40:46.867415 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:46.867451 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:46.867562 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:46.867836 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:46.867839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:46.868226 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:46.868401 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:46.868411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.869530 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.869537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:46.871483 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:46.871503 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.872798 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.872806 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:46.873032 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:46.873739 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:46.873772 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:46.877603 (+ 3831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":422,"threads_started":1}
Aug 08 23:40:46 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:46 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:46.940958 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:46.932148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:49291)
0808 23:40:46.932671 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:46.932687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:46.932843 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:46.933172 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:46.933176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:46.933619 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:46.933778 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:46.933786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.934843 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.934849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:46.935872 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:46.935884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.936590 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.936595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:46.936831 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:46.937521 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:46.937544 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:46.940727 (+ 3183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":287,"threads_started":1}
W20260808 23:40:46.941241 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:46.932207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:49291 (local address 127.18.186.190:36651)
0808 23:40:46.932408 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:46.932414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:46.932772 (+ 358us) server_negotiation.cc:407] Connection header received
0808 23:40:46.932974 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:46.932978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:46.933037 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:46.933142 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:46.933944 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.934695 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.936016 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:46.936432 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:46.936574 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:46.940966 (+ 4392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:49291: BlockingRecv error: recv got EOF from 127.18.186.129:49291 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 23:40:47 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:47 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:47.386798 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:47.378455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:33271)
0808 23:40:47.379026 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:47.379042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:47.379202 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:47.379660 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:47.379665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:47.380075 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:47.380220 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:47.380229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.381207 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.381213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:47.382109 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:47.382120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.382848 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.382854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:47.383023 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:47.383701 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:47.383719 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:47.386485 (+ 2766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":300,"threads_started":1}
W20260808 23:40:47.386933 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:47.378552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:33271 (local address 127.18.186.190:36651)
0808 23:40:47.378730 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:47.378737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:47.379130 (+ 393us) server_negotiation.cc:407] Connection header received
0808 23:40:47.379374 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:47.379379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:47.379509 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:47.379623 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:47.380362 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.381066 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.382252 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.382675 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.382806 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:47.386692 (+ 3886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:33271: BlockingRecv error: recv got EOF from 127.18.186.130:33271 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 23:40:47 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:47 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:47.893184 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:47.879158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:37373)
0808 23:40:47.879827 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:47.879867 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:47.880013 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:47.880413 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:47.880420 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:47.881000 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:47.881161 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:47.881170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.882200 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.882204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:47.883111 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:47.883122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.883870 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.883874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:47.884014 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:47.889402 (+ 5388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:47.889421 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:47.892879 (+ 3458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":414,"threads_started":1}
W20260808 23:40:47.893381 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:47.879260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:37373 (local address 127.18.186.190:36651)
0808 23:40:47.879509 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:47.879515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:47.879964 (+ 449us) server_negotiation.cc:407] Connection header received
0808 23:40:47.880163 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:47.880168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:47.880257 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:47.880356 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:47.881290 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.882082 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.883244 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.883726 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.883815 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:47.893143 (+ 9328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:37373: BlockingRecv error: recv got EOF from 127.18.186.131:37373 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 23:40:47 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:47 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:47.950685 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:47.941827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:34883)
0808 23:40:47.942524 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:47.942542 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:47.942703 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:47.943084 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:47.943092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:47.943598 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:47.943771 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:47.943781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.944839 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.944845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:47.945765 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:47.945777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.946544 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.946551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:47.946724 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:47.947342 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:47.947359 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:47.950327 (+ 2968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":304,"threads_started":1}
W20260808 23:40:47.950831 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:47.941927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:34883 (local address 127.18.186.190:36651)
0808 23:40:47.942193 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:47.942201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:47.942630 (+ 429us) server_negotiation.cc:407] Connection header received
0808 23:40:47.942828 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:47.942833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:47.942919 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:47.943030 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:47.943924 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.944681 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.945931 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:47.946387 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:47.946527 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:47.950585 (+ 4058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:34883: BlockingRecv error: recv got EOF from 127.18.186.129:34883 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 23:40:48 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:48 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:48.396517 29204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:48.387824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:57569)
0808 23:40:48.388449 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:48.388466 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:48.388580 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:48.388912 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:48.388915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:48.389324 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:48.389492 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:48.389500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.390453 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.390458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:48.391968 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:48.391980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.392610 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.392614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:48.392719 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:48.393278 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:48.393295 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:48.396252 (+ 2957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":342,"threads_started":1}
W20260808 23:40:48.396706 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:48.387904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:57569 (local address 127.18.186.190:36651)
0808 23:40:48.388165 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:48.388175 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:48.388548 (+ 373us) server_negotiation.cc:407] Connection header received
0808 23:40:48.388696 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:48.388700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:48.388769 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:48.388862 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:48.389610 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.390327 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.392092 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.392486 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.392584 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:48.396451 (+ 3867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:57569: BlockingRecv error: recv got EOF from 127.18.186.130:57569 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 23:40:48 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:48 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:48.902973 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:48.894072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:38985)
0808 23:40:48.894660 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:48.894677 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:48.894800 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:48.895093 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:48.895096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:48.895598 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:48.895770 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:48.895779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.896719 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.896722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:48.897847 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:48.897860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.898555 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.898560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:48.898671 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:48.899233 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:48.899251 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:48.902607 (+ 3356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":251,"threads_started":1}
W20260808 23:40:48.903029 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:48.894132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:38985 (local address 127.18.186.190:36651)
0808 23:40:48.894358 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:48.894367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:48.894755 (+ 388us) server_negotiation.cc:407] Connection header received
0808 23:40:48.894902 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:48.894908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:48.894971 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:48.895053 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:48.895892 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.896600 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.897985 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.898426 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.898520 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:48.902799 (+ 4279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:38985: BlockingRecv error: recv got EOF from 127.18.186.131:38985 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 23:40:48 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:48 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:48.959231 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:48.951538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:42257)
0808 23:40:48.952080 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:48.952096 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:48.952211 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:48.952497 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:48.952500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:48.952904 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:48.953058 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:48.953067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.954045 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.954049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:48.954962 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:48.954973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.955671 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.955676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:48.955783 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:48.956352 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:48.956369 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:48.959001 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":290,"threads_started":1}
W20260808 23:40:48.959410 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:48.951696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:42257 (local address 127.18.186.190:36651)
0808 23:40:48.951866 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:48.951870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:48.952171 (+ 301us) server_negotiation.cc:407] Connection header received
0808 23:40:48.952307 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:48.952311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:48.952372 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:48.952466 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:48.953181 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.953923 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.955083 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:48.955522 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:48.955613 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:48.959187 (+ 3574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:42257: BlockingRecv error: recv got EOF from 127.18.186.129:42257 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 23:40:49 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:49 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:49.406311 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:49.397371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:38389)
0808 23:40:49.397900 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:49.397915 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:49.398027 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:49.398323 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:49.398327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:49.398714 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:49.398859 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:49.398867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.400012 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.400016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:49.401408 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:49.401422 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.402149 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.402156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:49.402310 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:49.403039 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:49.403057 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:49.406080 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":282,"threads_started":1}
W20260808 23:40:49.406522 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:49.397494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:38389 (local address 127.18.186.190:36651)
0808 23:40:49.397660 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:49.397664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:49.397988 (+ 324us) server_negotiation.cc:407] Connection header received
0808 23:40:49.398124 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:49.398127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:49.398201 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:49.398279 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:49.398981 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.399856 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.401528 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.401974 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.402068 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:49.406294 (+ 4226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:38389: BlockingRecv error: recv got EOF from 127.18.186.130:38389 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 23:40:49 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:49 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:49.911823 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:49.903934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:55953)
0808 23:40:49.904426 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:49.904443 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:49.904557 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:49.904842 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:49.904845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:49.905249 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:49.905414 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:49.905423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.906354 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.906357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:49.907261 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:49.907271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.908081 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.908086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:49.908197 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:49.908746 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:49.908762 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:49.911545 (+ 2783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":255,"threads_started":1}
W20260808 23:40:49.912074 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:49.904035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:55953 (local address 127.18.186.190:36651)
0808 23:40:49.904292 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:49.904298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:49.904523 (+ 225us) server_negotiation.cc:407] Connection header received
0808 23:40:49.904652 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:49.904656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:49.904715 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:49.904805 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:49.905546 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.906227 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.907389 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.907926 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.908019 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:49.911792 (+ 3773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:55953: BlockingRecv error: recv got EOF from 127.18.186.131:55953 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 08 23:40:49 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:49 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:49.968990 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:49.960407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:47275)
0808 23:40:49.961330 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:49.961354 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:49.961515 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:49.961934 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:49.961938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:49.962316 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:49.962516 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:49.962524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.963546 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.963549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:49.964482 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:49.964493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.965170 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.965174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:49.965290 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:49.965874 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:49.965891 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:49.968728 (+ 2837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":652,"threads_started":1}
W20260808 23:40:49.969211 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:49.960500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:47275 (local address 127.18.186.190:36651)
0808 23:40:49.960677 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:49.960685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:49.961454 (+ 769us) server_negotiation.cc:407] Connection header received
0808 23:40:49.961699 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:49.961704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:49.961783 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:49.961863 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:49.962649 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.963387 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.964604 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:49.965040 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:49.965134 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:49.968953 (+ 3819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:47275: BlockingRecv error: recv got EOF from 127.18.186.129:47275 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 23:40:50 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:50 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:50.417104 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:50.407506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:60889)
0808 23:40:50.408264 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:50.408280 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:50.408451 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:50.408828 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:50.408834 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:50.409262 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:50.409433 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:50.409442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.410496 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.410502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:50.411546 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:50.411557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.412420 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.412425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:50.412612 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:50.413375 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:50.413399 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:50.416727 (+ 3328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":493,"threads_started":1}
W20260808 23:40:50.417196 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:50.407583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:60889 (local address 127.18.186.190:36651)
0808 23:40:50.407821 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:50.407831 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:50.408377 (+ 546us) server_negotiation.cc:407] Connection header received
0808 23:40:50.408592 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:50.408598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:50.408688 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:50.408793 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:50.409587 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.410351 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.411679 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.412211 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.412351 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:50.416946 (+ 4595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:60889: BlockingRecv error: recv got EOF from 127.18.186.130:60889 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 23:40:50 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:50 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:50.921974 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:50.913049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:36967)
0808 23:40:50.913573 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:50.913589 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:50.913746 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:50.914100 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:50.914104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:50.914612 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:50.914850 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:50.914866 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.916061 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.916067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:50.917166 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:50.917177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.917872 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.917879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:50.918084 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:50.918669 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:50.918689 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:50.921720 (+ 3031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":273,"threads_started":1}
W20260808 23:40:50.922262 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:50.913108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:36967 (local address 127.18.186.190:36651)
0808 23:40:50.913344 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:50.913353 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:50.913676 (+ 323us) server_negotiation.cc:407] Connection header received
0808 23:40:50.913870 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:50.913875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:50.913948 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:50.914079 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:50.915007 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.915921 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.917309 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.917710 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.917850 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:50.921964 (+ 4114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:36967: BlockingRecv error: recv got EOF from 127.18.186.131:36967 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 23:40:50 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:50 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:50.978710 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:50.969949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:45473)
0808 23:40:50.970496 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:50.970512 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:50.970622 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:50.970891 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:50.970894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:50.971256 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:50.971520 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:50.971537 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.972571 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.972574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:50.973496 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:50.973507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.974619 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.974627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:50.974801 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:50.975571 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:50.975589 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:50.978496 (+ 2907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":309,"threads_started":1}
W20260808 23:40:50.978988 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:50.970158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:45473 (local address 127.18.186.190:36651)
0808 23:40:50.970340 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:50.970344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:50.970586 (+ 242us) server_negotiation.cc:407] Connection header received
0808 23:40:50.970703 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:50.970706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:50.970764 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:50.970846 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:50.971656 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.972451 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.973655 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:50.974367 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:50.974557 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:50.978680 (+ 4123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:45473: BlockingRecv error: recv got EOF from 127.18.186.129:45473 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 23:40:51 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:51 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:51.427157 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:51.418157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:49965)
0808 23:40:51.418689 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:51.418705 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:51.418855 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:51.419265 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:51.419270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:51.419694 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:51.419846 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:51.419854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.420847 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.420857 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:51.421836 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:51.421847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.422618 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.422624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:51.422799 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:51.423449 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:51.423467 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:51.426825 (+ 3358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":308,"threads_started":1}
W20260808 23:40:51.427286 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:51.418227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:49965 (local address 127.18.186.190:36651)
0808 23:40:51.418464 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:51.418472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:51.418797 (+ 325us) server_negotiation.cc:407] Connection header received
0808 23:40:51.419012 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:51.419017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:51.419120 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:51.419237 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:51.419999 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.420704 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.421979 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.422451 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.422586 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:51.427040 (+ 4454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:49965: BlockingRecv error: recv got EOF from 127.18.186.130:49965 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 23:40:51 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:51 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:51.931563 29215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:51.922985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:60823)
0808 23:40:51.923546 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:51.923563 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:51.923674 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:51.924045 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:51.924048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:51.924419 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:51.924567 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:51.924575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.925909 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.925913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:51.926847 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:51.926858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.927606 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.927615 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:51.927822 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:51.928469 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:51.928486 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:51.931208 (+ 2722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":288,"threads_started":1}
W20260808 23:40:51.931694 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:51.923191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:60823 (local address 127.18.186.190:36651)
0808 23:40:51.923493 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:51.923498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:51.923637 (+ 139us) server_negotiation.cc:407] Connection header received
0808 23:40:51.923803 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:51.923806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:51.923867 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:51.923966 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:51.924751 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.925779 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.926970 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.927377 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.927574 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:51.931420 (+ 3846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:60823: BlockingRecv error: recv got EOF from 127.18.186.131:60823 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 08 23:40:51 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:51 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:51.988580 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:51.979836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:51969)
0808 23:40:51.980357 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:51.980373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:51.980498 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:51.980822 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:51.980825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:51.981255 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:51.981436 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:51.981446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.982390 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.982393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:51.983408 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:51.983443 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.984451 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.984457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:51.984637 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:51.985238 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:51.985255 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:51.988307 (+ 3052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":298,"threads_started":1}
W20260808 23:40:51.988760 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:51.979891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:51969 (local address 127.18.186.190:36651)
0808 23:40:51.980106 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:51.980111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:51.980450 (+ 339us) server_negotiation.cc:407] Connection header received
0808 23:40:51.980593 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:51.980597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:51.980688 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:51.980771 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:51.981563 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.982277 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.983558 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:51.984270 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:51.984395 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:51.988532 (+ 4137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:51969: BlockingRecv error: recv got EOF from 127.18.186.129:51969 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 23:40:52 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:52 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:52.437542 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:52.428181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:38077)
0808 23:40:52.428696 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:52.428712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:52.428865 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:52.429259 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:52.429264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:52.429705 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:52.429875 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:52.429891 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.430939 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.430945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:52.432103 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:52.432120 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.432814 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.432819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:52.432973 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:52.433643 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:52.433661 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:52.437191 (+ 3530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":281,"threads_started":1}
W20260808 23:40:52.437688 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:52.428299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:38077 (local address 127.18.186.190:36651)
0808 23:40:52.428548 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:52.428556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:52.428797 (+ 241us) server_negotiation.cc:407] Connection header received
0808 23:40:52.429022 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:52.429027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:52.429117 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:52.429223 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:52.430028 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.430788 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.432263 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.432674 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.432783 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:52.437398 (+ 4615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:38077: BlockingRecv error: recv got EOF from 127.18.186.130:38077 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 23:40:52 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:52 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:52.946205 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:52.932587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:44101)
0808 23:40:52.933142 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:52.933158 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:52.933317 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:52.933688 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:52.933692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:52.934131 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:52.934301 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:52.934310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.935372 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.935379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:52.936390 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:52.936403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.937108 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.937114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:52.937302 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:52.942824 (+ 5522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:52.942843 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:52.945831 (+ 2988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":306,"threads_started":1}
W20260808 23:40:52.946326 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:52.932673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:44101 (local address 127.18.186.190:36651)
0808 23:40:52.932933 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:52.932941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:52.933247 (+ 306us) server_negotiation.cc:407] Connection header received
0808 23:40:52.933476 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:52.933485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:52.933550 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:52.933659 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:52.934460 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.935238 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.936532 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.936930 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.937066 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:52.946088 (+ 9022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:44101: BlockingRecv error: recv got EOF from 127.18.186.131:44101 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 23:40:52 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:52 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:52.998250 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:52.989463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:53807)
0808 23:40:52.989956 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:52.989973 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:52.990132 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:52.990424 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:52.990427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:52.990915 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:52.991086 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:52.991095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.992136 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.992140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:52.993044 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:52.993073 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.993811 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.993817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:52.993969 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:52.994723 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:52.994743 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:52.997914 (+ 3171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":288,"threads_started":1}
W20260808 23:40:52.998360 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:52.989619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:53807 (local address 127.18.186.190:36651)
0808 23:40:52.989854 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:52.989861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:52.990064 (+ 203us) server_negotiation.cc:407] Connection header received
0808 23:40:52.990232 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:52.990235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:52.990295 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:52.990371 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:52.991216 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.991982 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.993187 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:52.993613 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:52.993703 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:52.998128 (+ 4425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:53807: BlockingRecv error: recv got EOF from 127.18.186.129:53807 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 08 23:40:53 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:53 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:53.447903 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:53.438697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:49099)
0808 23:40:53.439214 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:53.439229 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:53.439403 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:53.439857 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:53.439862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:53.440287 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:53.440444 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:53.440452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:53.441616 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:53.441622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:53.442770 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:53.442788 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:53.443655 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:53.443659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:53.443793 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:53.444421 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:53.444440 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:53.447556 (+ 3116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":303,"threads_started":1}
W20260808 23:40:53.448009 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:53.438741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:49099 (local address 127.18.186.190:36651)
0808 23:40:53.439004 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:53.439013 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:53.439317 (+ 304us) server_negotiation.cc:407] Connection header received
0808 23:40:53.439589 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:53.439594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:53.439689 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:53.439802 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:53.440591 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:53.441470 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:53.442944 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:53.443504 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:53.443645 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:53.447777 (+ 4132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:49099: BlockingRecv error: recv got EOF from 127.18.186.130:49099 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 23:40:53 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:53 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:53.955307 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:53.947192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:37769)
0808 23:40:53.947732 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:53.947748 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:53.947859 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:53.948168 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:53.948171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:53.948569 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:53.948727 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:53.948736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:53.949723 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:53.949726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:53.950655 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:53.950666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:53.951349 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:53.951353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:53.951573 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:53.952249 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:53.952266 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:53.955075 (+ 2809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":342,"threads_started":1}
W20260808 23:40:53.955557 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:53.947286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:37769 (local address 127.18.186.190:36651)
0808 23:40:53.947578 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:53.947585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:53.947831 (+ 246us) server_negotiation.cc:407] Connection header received
0808 23:40:53.947969 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:53.947973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:53.948035 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:53.948117 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:53.948880 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:53.949594 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:53.950796 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:53.951220 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:53.951332 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:53.955269 (+ 3937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:37769: BlockingRecv error: recv got EOF from 127.18.186.131:37769 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 08 23:40:54 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:54 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:54.007849 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:53.999334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:59607)
0808 23:40:53.999965 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:53.999981 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:54.000100 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:54.000405 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:54.000409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:54.000795 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:54.000986 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:54.000996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.001994 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.001997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:54.002923 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:54.002935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.003682 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.003687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:54.003871 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:54.004530 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:54.004548 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:54.007600 (+ 3052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":345,"threads_started":1}
W20260808 23:40:54.008181 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:53.999446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:59607 (local address 127.18.186.190:36651)
0808 23:40:53.999620 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:53.999625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:54.000152 (+ 527us) server_negotiation.cc:407] Connection header received
0808 23:40:54.000212 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:54.000215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:54.000277 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:54.000361 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:54.001125 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.001845 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.003060 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.003502 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.003642 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:54.007941 (+ 4299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:59607: BlockingRecv error: recv got EOF from 127.18.186.129:59607 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 23:40:54 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:54 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:54.457808 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:54.448927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:40109)
0808 23:40:54.449552 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:54.449567 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:54.449722 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:54.450070 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:54.450075 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:54.450490 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:54.450643 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:54.450651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.451780 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.451787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:54.452713 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:54.452723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.453510 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.453517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:54.453681 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:54.454343 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:54.454361 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:54.457428 (+ 3067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":334,"threads_started":1}
W20260808 23:40:54.458026 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:54.449135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:40109 (local address 127.18.186.190:36651)
0808 23:40:54.449383 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:54.449392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:54.449650 (+ 258us) server_negotiation.cc:407] Connection header received
0808 23:40:54.449850 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:54.449855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:54.449932 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:54.450036 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:54.450791 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.451649 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.452867 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.453311 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.453449 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:54.457742 (+ 4293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:40109: BlockingRecv error: recv got EOF from 127.18.186.130:40109 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 08 23:40:54 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:54 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:54.965065 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:54.956363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:35205)
0808 23:40:54.956981 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:54.956998 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:54.957111 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:54.957393 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:54.957396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:54.957794 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:54.957984 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:54.957994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.959234 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.959239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:54.960270 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:54.960283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.961153 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.961158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:54.961284 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:54.961891 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:54.961908 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:54.964797 (+ 2889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":298,"threads_started":1}
W20260808 23:40:54.965161 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:54.956425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:35205 (local address 127.18.186.190:36651)
0808 23:40:54.956600 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:54.956605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:54.957068 (+ 463us) server_negotiation.cc:407] Connection header received
0808 23:40:54.957204 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:54.957207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:54.957267 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:54.957349 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:54.958131 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.959079 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.960416 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:54.960984 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:54.961077 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:54.964978 (+ 3901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:35205: BlockingRecv error: recv got EOF from 127.18.186.131:35205 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 23:40:55 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:55 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:55.017462 29228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:55.008855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:47715)
0808 23:40:55.009365 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:55.009381 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:55.009536 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:55.009881 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:55.009886 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:55.010322 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:55.010502 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:55.010513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.011587 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.011592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:55.012691 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:55.012703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.013380 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.013386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:55.013561 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:55.014180 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:55.014196 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:55.017192 (+ 2996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":270,"threads_started":1}
W20260808 23:40:55.017638 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:55.008936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:47715 (local address 127.18.186.190:36651)
0808 23:40:55.009230 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:55.009237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:55.009467 (+ 230us) server_negotiation.cc:407] Connection header received
0808 23:40:55.009667 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:55.009672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:55.009740 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:55.009845 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:55.010651 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.011386 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.012833 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.013237 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.013370 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:55.017394 (+ 4024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:47715: BlockingRecv error: recv got EOF from 127.18.186.129:47715 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
Aug 08 23:40:55 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:55 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:55.467568 29229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:55.459083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:51765)
0808 23:40:55.459619 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:55.459660 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:55.459810 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:55.460260 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:55.460265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:55.460688 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:55.460887 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:55.460896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.461877 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.461880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:55.462770 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:55.462780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.463522 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.463530 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:55.463670 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:55.464247 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:55.464264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:55.467164 (+ 2900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":280,"threads_started":1}
W20260808 23:40:55.467669 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:55.459131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:51765 (local address 127.18.186.190:36651)
0808 23:40:55.459321 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:55.459328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:55.459763 (+ 435us) server_negotiation.cc:407] Connection header received
0808 23:40:55.459988 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:55.459993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:55.460080 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:55.460182 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:55.461037 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.461760 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.462910 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.463333 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.463453 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:55.467407 (+ 3954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:51765: BlockingRecv error: recv got EOF from 127.18.186.130:51765 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 23:40:55 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:55 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.131@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:55.974939 29230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:55.966119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.131:35997)
0808 23:40:55.966645 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:55.966662 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:55.966774 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:55.967078 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:55.967081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:55.967541 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:55.967724 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:55.967733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.969140 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.969147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:55.970130 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:55.970144 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.970897 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.970903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:55.971085 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:55.971821 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:55.971839 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:55.974576 (+ 2737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":292,"threads_started":1}
W20260808 23:40:55.975037 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:55.966263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:35997 (local address 127.18.186.190:36651)
0808 23:40:55.966478 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:55.966482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:55.966738 (+ 256us) server_negotiation.cc:407] Connection header received
0808 23:40:55.966874 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:55.966878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:55.966949 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:55.967036 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:55.967946 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.968924 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.970292 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:55.970728 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:55.970870 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:55.974780 (+ 3910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.131:35997: BlockingRecv error: recv got EOF from 127.18.186.131:35997 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 23:40:56 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:56 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.129@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:56.027412 29231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:56.018589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.129:38021)
0808 23:40:56.019181 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:56.019197 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:56.019366 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:56.019801 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:56.019807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:56.020221 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:56.020420 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:56.020429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.021509 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.021514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:56.022438 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:56.022450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.023179 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.023185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:56.023360 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:56.024053 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:56.024070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:56.027003 (+ 2933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":322,"threads_started":1}
W20260808 23:40:56.027519 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:56.018673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:38021 (local address 127.18.186.190:36651)
0808 23:40:56.019012 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:56.019020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:56.019289 (+ 269us) server_negotiation.cc:407] Connection header received
0808 23:40:56.019526 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:56.019534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:56.019630 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:56.019748 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:56.020563 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.021359 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.022590 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.023016 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.023154 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:56.027211 (+ 4057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.129:38021: BlockingRecv error: recv got EOF from 127.18.186.129:38021 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 08 23:40:56 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:40:56 dist-test-slave-5k54 krb5kdc[28074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.186.130@KRBTEST.COM for oryx/127.18.186.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:40:56.478171 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:56.468785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:36651 (local address 127.18.186.130:51049)
0808 23:40:56.469424 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:56.469440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:56.469602 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:56.469986 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:56.469991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:56.470432 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:56.470595 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:56.470604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.471861 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.471867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:56.472766 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:56.472777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.473516 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.473522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:56.473689 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:56.474558 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:40:56.474576 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:56.477774 (+ 3198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.186.190:36651: Server oryx/127.18.186.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":353,"threads_started":1}
W20260808 23:40:56.478246 28882 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:56.468975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:51049 (local address 127.18.186.190:36651)
0808 23:40:56.469228 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:56.469237 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:56.469531 (+ 294us) server_negotiation.cc:407] Connection header received
0808 23:40:56.469734 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:56.469740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:56.469817 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:56.469934 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:56.470738 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.471706 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.472919 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.473361 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.473494 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:56.477993 (+ 4499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.186.130:51049: BlockingRecv error: recv got EOF from 127.18.186.130:51049 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 23:40:56.795526 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28718
I20260808 23:40:56.808658 29234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:56.796821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.18.186.129:36733)
0808 23:40:56.797467 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:56.797475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:56.797502 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:40:56.797562 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:56.797593 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:56.797871 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:56.798051 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:56.799646 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.800744 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.802064 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.802653 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.802765 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:56.805485 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:56.805494 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:56.805508 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:56.805547 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:56.808106 (+ 2559us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.186.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 23:40:56.808386 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60008: Request ticket server kudu/127.18.186.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":345,"threads_started":1}
W20260808 23:40:56.808779 29234 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60008: Request ticket server kudu/127.18.186.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 23:40:56.808899 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28718: Not authorized: Client connection negotiation failed: client connection to 127.18.186.129:36733: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.186.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 23:40:56.809140 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 28718
I20260808 23:40:56.828902 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28885
I20260808 23:40:56.841866 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:56.830058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.18.186.130:44931)
0808 23:40:56.830622 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:56.830630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:56.830661 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:40:56.830721 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:56.830728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:56.830955 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:56.831171 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:56.832599 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.833656 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.834931 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.835650 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.835795 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:56.838421 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:56.838430 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:56.838466 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:56.838517 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:56.841306 (+ 2789us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.186.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 23:40:56.841595 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46980: Request ticket server kudu/127.18.186.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":268,"threads_started":1}
W20260808 23:40:56.841993 29236 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46980: Request ticket server kudu/127.18.186.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 23:40:56.842088 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28885: Not authorized: Client connection negotiation failed: client connection to 127.18.186.130:44931: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.186.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 23:40:56.842258 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 28885
I20260808 23:40:56.862445 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 29030
I20260808 23:40:56.877103 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:56.863576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41978 (local address 127.18.186.131:44089)
0808 23:40:56.864277 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:56.864284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:56.864314 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:40:56.864378 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:56.864385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:56.864620 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:56.864789 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:56.866146 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.867162 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.868551 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:56.869417 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:56.869578 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:56.872471 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:56.872485 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:56.872505 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:40:56.872567 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:56.876388 (+ 3821us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.186.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 23:40:56.876734 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41978: Request ticket server kudu/127.18.186.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":315,"threads_started":1}
W20260808 23:40:56.877120 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 29030: Not authorized: Client connection negotiation failed: client connection to 127.18.186.131:44089: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.186.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 23:40:56.877236 29237 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41978: Request ticket server kudu/127.18.186.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 23:40:56.877318 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 29030
I20260808 23:40:56.899617 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 28641
I20260808 23:40:56.901028 28710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:57.021591 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 28641
2026-08-08T23:40:57Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21287 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T23:40:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:57Z Disabled control of system clock
I20260808 23:40:57.108687 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:46229
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:46229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 23:40:57.438114 29243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:57.438402 29243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:57.438542 29243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:57.445804 29243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:57.445910 29243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:57.445948 29243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:57.446096 29243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:57.446182 29243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:57.455295 29243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:46229
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:46229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:57.456646 29243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:57.458484 29243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:57.470949 29253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:57.470983 29251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:57.471101 29250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:57.471695 29243 server_base.cc:1034] running on GCE node
I20260808 23:40:57.472504 29243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:57.473891 29243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:57.475132 29243 hybrid_clock.cc:648] HybridClock initialized: now 1786232457475081 us; error 48 us; skew 500 ppm
I20260808 23:40:57.475561 29243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:57.477888 29243 webserver.cc:460] Webserver started at http://127.18.186.190:44309/ using document root <none> and password file <none>
I20260808 23:40:57.478597 29243 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:57.478730 29243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:57.479080 29243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:57.482048 29243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "488ae521ee8848609e425fc26e4eebea"
format_stamp: "Formatted at 2026-08-08 23:40:57 on dist-test-slave-5k54"
server_key: "4ea0a99f536c71e656fdd78c1e95b73e"
server_key_iv: "f4eae8b23607c7ed3fddba8392cbfadd"
server_key_version: "encryptionkey@0"
I20260808 23:40:57.482843 29243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "488ae521ee8848609e425fc26e4eebea"
format_stamp: "Formatted at 2026-08-08 23:40:57 on dist-test-slave-5k54"
server_key: "4ea0a99f536c71e656fdd78c1e95b73e"
server_key_iv: "f4eae8b23607c7ed3fddba8392cbfadd"
server_key_version: "encryptionkey@0"
I20260808 23:40:57.487886 29243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 23:40:57.491246 29260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:57.492694 29243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:40:57.492897 29243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "488ae521ee8848609e425fc26e4eebea"
format_stamp: "Formatted at 2026-08-08 23:40:57 on dist-test-slave-5k54"
server_key: "4ea0a99f536c71e656fdd78c1e95b73e"
server_key_iv: "f4eae8b23607c7ed3fddba8392cbfadd"
server_key_version: "encryptionkey@0"
I20260808 23:40:57.493121 29243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:57.508131 29243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:57.509021 29243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:57.509361 29243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:57.530143 29243 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:46229
I20260808 23:40:57.530149 29318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:46229 every 8 connection(s)
I20260808 23:40:57.531750 29243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:40:57.532938 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 29243
I20260808 23:40:57.533145 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:40:57.534000 19178 external_mini_cluster.cc:1428] Setting key 648a83b579465bcc7cd7fda634bf9d14
I20260808 23:40:57.536681 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:40:57.548153 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:57.536212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.18.186.190:46229)
0808 23:40:57.537233 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:57.537243 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:57.537285 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:40:57.538894 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:57.538915 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:57.539374 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:57.539762 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:57.540459 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:57.541795 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:57.543697 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:57.544370 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:57.544498 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:57.545821 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:57.545828 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:57.545840 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:57.546389 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:57.546464 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:57.546488 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:57.546583 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:57.546835 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 23:40:57.547188 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":379,"threads_started":1}
I20260808 23:40:57.549412 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea: Bootstrap starting.
I20260808 23:40:57.552640 29319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea: Neither blocks nor log segments found. Creating new log.
I20260808 23:40:57.553632 29319 log.cc:824] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea: Log is configured to *not* fsync() on all Append() calls
I20260808 23:40:57.556694 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea: No bootstrap required, opened a new log
I20260808 23:40:57.563827 29319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488ae521ee8848609e425fc26e4eebea" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46229 } }
I20260808 23:40:57.564158 29319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:40:57.564260 29319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 488ae521ee8848609e425fc26e4eebea, State: Initialized, Role: FOLLOWER
I20260808 23:40:57.564842 29319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488ae521ee8848609e425fc26e4eebea" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46229 } }
I20260808 23:40:57.565078 29319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:40:57.565215 29319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:40:57.565407 29319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:40:57.567483 29319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488ae521ee8848609e425fc26e4eebea" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46229 } }
I20260808 23:40:57.567968 29319 leader_election.cc:302] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 488ae521ee8848609e425fc26e4eebea; no voters:
I20260808 23:40:57.568580 29319 leader_election.cc:288] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:40:57.568739 29325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:40:57.570147 29325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [term 1 LEADER]: Becoming Leader. State: Replica: 488ae521ee8848609e425fc26e4eebea, State: Running, Role: LEADER
I20260808 23:40:57.570665 29325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488ae521ee8848609e425fc26e4eebea" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46229 } }
I20260808 23:40:57.570983 29319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:40:57.573786 29327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 488ae521ee8848609e425fc26e4eebea. Latest consensus state: current_term: 1 leader_uuid: "488ae521ee8848609e425fc26e4eebea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488ae521ee8848609e425fc26e4eebea" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46229 } } }
I20260808 23:40:57.574182 29327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [sys.catalog]: This master's current role is: LEADER
I20260808 23:40:57.574757 29326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "488ae521ee8848609e425fc26e4eebea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "488ae521ee8848609e425fc26e4eebea" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46229 } } }
I20260808 23:40:57.575271 29326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea [sys.catalog]: This master's current role is: LEADER
W20260808 23:40:57.582078 29342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:40:57.582192 29342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:40:57.582366 29334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:40:57.584504 29334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:40:57.593556 29334 catalog_manager.cc:1347] Generated new cluster ID: 82644995df7949b884d480ede82611ca
I20260808 23:40:57.593667 29334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:40:57.606716 29334 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:40:57.608251 29334 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:40:57.613673 29334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 488ae521ee8848609e425fc26e4eebea: Generated new TSK 0
I20260808 23:40:57.614456 29334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:40:57.621677 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:46229
--builtin_ntp_servers=127.18.186.148:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 23:40:57.977512 29345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:57.977793 29345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:57.977885 29345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:57.985072 29345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:57.985173 29345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:57.985276 29345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:40:57.994181 29345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:46229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:57.995393 29345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:57.997421 29345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:58.011013 29350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:58.011018 29353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:58.011014 29351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:58.011660 29345 server_base.cc:1034] running on GCE node
I20260808 23:40:58.012302 29345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:58.013248 29345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:58.014458 29345 hybrid_clock.cc:648] HybridClock initialized: now 1786232458014401 us; error 42 us; skew 500 ppm
I20260808 23:40:58.014796 29345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:58.022401 29345 webserver.cc:460] Webserver started at http://127.18.186.129:41909/ using document root <none> and password file <none>
I20260808 23:40:58.023137 29345 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:58.023259 29345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:58.023665 29345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:58.026588 29345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "6fff2c523be2463dbfbdfe1092034eb0"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "960e3d60f18ebdce35cd331c2d71d533"
server_key_iv: "0f594238ae809a76a1a551e1c8c75ee2"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.027488 29345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "6fff2c523be2463dbfbdfe1092034eb0"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "960e3d60f18ebdce35cd331c2d71d533"
server_key_iv: "0f594238ae809a76a1a551e1c8c75ee2"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.033061 29345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 23:40:58.036643 29359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:58.037984 29345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:40:58.038159 29345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "6fff2c523be2463dbfbdfe1092034eb0"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "960e3d60f18ebdce35cd331c2d71d533"
server_key_iv: "0f594238ae809a76a1a551e1c8c75ee2"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.038399 29345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:58.069079 29345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:58.069958 29345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:58.070209 29345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:58.071148 29345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:58.072767 29345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:58.072856 29345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:58.072948 29345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:58.073004 29345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:58.106073 29345 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:44667
I20260808 23:40:58.106174 29482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:44667 every 8 connection(s)
I20260808 23:40:58.107417 29345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:40:58.113762 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 29345
I20260808 23:40:58.113974 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:40:58.114651 19178 external_mini_cluster.cc:1428] Setting key bc24174adba497e41fe71936075bff19
I20260808 23:40:58.117622 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:46229
--builtin_ntp_servers=127.18.186.148:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 23:40:58.121381 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:58.110185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:34509 (local address 127.18.186.190:46229)
0808 23:40:58.110630 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:58.110635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:58.111961 (+ 1326us) server_negotiation.cc:407] Connection header received
0808 23:40:58.113841 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:58.113846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:58.113955 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:58.114082 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:58.115356 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:58.116460 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:58.118127 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:58.118928 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:58.119084 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:58.120411 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:58.120419 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:58.120426 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:58.120450 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:58.120499 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:58.120517 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:58.120609 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:58.120821 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 23:40:58.120926 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":340,"thread_start_us":233,"threads_started":1}
I20260808 23:40:58.122072 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:58.110843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:46229 (local address 127.18.186.129:34509)
0808 23:40:58.111806 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:58.111848 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:58.113566 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:58.114415 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:58.114423 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:58.114507 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:58.115123 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:58.115141 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:58.116646 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:58.116651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:58.117981 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:58.117994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:58.119134 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:58.119139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:58.119331 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:58.120095 (+ 764us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:58.120098 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:58.120134 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:58.120216 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:58.120612 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:58.120620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:58.120625 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:40:58.120757 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 23:40:58.121023 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":262,"threads_started":1}
I20260808 23:40:58.123642 29483 heartbeater.cc:347] Connected to a master server at 127.18.186.190:46229
I20260808 23:40:58.123963 29483 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:58.124883 29483 heartbeater.cc:511] Master 127.18.186.190:46229 requested a full tablet report, sending...
I20260808 23:40:58.127027 29279 ts_manager.cc:194] Registered new tserver with Master: 6fff2c523be2463dbfbdfe1092034eb0 (127.18.186.129:44667)
I20260808 23:40:58.128971 29279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.129:34509
W20260808 23:40:58.452757 29487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:58.453095 29487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:58.453269 29487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:58.460438 29487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:58.460570 29487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:58.460752 29487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:40:58.470108 29487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:46229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:58.471534 29487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:58.473289 29487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:58.486358 29493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:58.486326 29496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:58.486431 29494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:58.490854 29487 server_base.cc:1034] running on GCE node
I20260808 23:40:58.491389 29487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:58.492309 29487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:58.493553 29487 hybrid_clock.cc:648] HybridClock initialized: now 1786232458493500 us; error 33 us; skew 500 ppm
I20260808 23:40:58.493947 29487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:58.496363 29487 webserver.cc:460] Webserver started at http://127.18.186.130:39705/ using document root <none> and password file <none>
I20260808 23:40:58.497139 29487 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:58.497267 29487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:58.497602 29487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:58.500519 29487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "cf5888a387934582a997845c7f3619e4"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "0626285b78ae2af557b73dea7e87f18c"
server_key_iv: "c1915ac104f0d19744706129f67172f6"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.501358 29487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "cf5888a387934582a997845c7f3619e4"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "0626285b78ae2af557b73dea7e87f18c"
server_key_iv: "c1915ac104f0d19744706129f67172f6"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.506506 29487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:40:58.509886 29503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:58.511189 29487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:40:58.511363 29487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "cf5888a387934582a997845c7f3619e4"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "0626285b78ae2af557b73dea7e87f18c"
server_key_iv: "c1915ac104f0d19744706129f67172f6"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.511603 29487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:58.530668 29487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:58.531553 29487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:58.531877 29487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:58.533185 29487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:58.534648 29487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:58.534780 29487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:58.534921 29487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:58.535012 29487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:58.571168 29487 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:42367
I20260808 23:40:58.571236 29628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:42367 every 8 connection(s)
I20260808 23:40:58.572691 29487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:40:58.576877 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 29487
I20260808 23:40:58.577054 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:40:58.577661 19178 external_mini_cluster.cc:1428] Setting key 2c0c0271528400df7d9d17c054addba6
I20260808 23:40:58.580691 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:46229
--builtin_ntp_servers=127.18.186.148:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 23:40:58.586802 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:58.575363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:56975 (local address 127.18.186.190:46229)
0808 23:40:58.575608 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:58.575614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:58.577231 (+ 1617us) server_negotiation.cc:407] Connection header received
0808 23:40:58.579532 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:58.579539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:58.579628 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:58.579723 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:58.580838 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:58.582042 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:58.583693 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:58.584392 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:58.584542 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:58.585892 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:58.585900 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:58.585907 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:58.585933 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:58.585974 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:58.585992 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:58.586088 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:58.586289 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 23:40:58.586401 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 23:40:58.587855 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:58.575967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:46229 (local address 127.18.186.130:56975)
0808 23:40:58.577058 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:58.577110 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:58.579171 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:58.580006 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:58.580014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:58.580100 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:58.580662 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:58.580679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:58.582187 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:58.582191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:58.583537 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:58.583555 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:58.584606 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:58.584611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:58.584838 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:58.585563 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:58.585566 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:58.585605 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:58.585680 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:58.586089 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:58.586096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:58.586102 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 23:40:58.586239 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 23:40:58.586551 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":253,"threads_started":1}
I20260808 23:40:58.589449 29629 heartbeater.cc:347] Connected to a master server at 127.18.186.190:46229
I20260808 23:40:58.589745 29629 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:58.590513 29629 heartbeater.cc:511] Master 127.18.186.190:46229 requested a full tablet report, sending...
I20260808 23:40:58.591967 29279 ts_manager.cc:194] Registered new tserver with Master: cf5888a387934582a997845c7f3619e4 (127.18.186.130:42367)
I20260808 23:40:58.592765 29279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.130:56975
W20260808 23:40:58.922115 29633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:58.922386 29633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:58.922538 29633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:58.929661 29633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:58.929759 29633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:58.929949 29633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:40:58.938659 29633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:46229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:58.940012 29633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:58.941840 29633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:58.954872 29638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:58.955077 29639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:58.955214 29641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:58.956126 29633 server_base.cc:1034] running on GCE node
I20260808 23:40:58.956701 29633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:58.957639 29633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:58.958841 29633 hybrid_clock.cc:648] HybridClock initialized: now 1786232458958794 us; error 40 us; skew 500 ppm
I20260808 23:40:58.959237 29633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:58.961572 29633 webserver.cc:460] Webserver started at http://127.18.186.131:35843/ using document root <none> and password file <none>
I20260808 23:40:58.962283 29633 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:58.962374 29633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:58.962708 29633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:58.965583 29633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "ea20a457f9f044d299810b6861fe7a82"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "7d7c2dbe66404477eff384ec2eca69c0"
server_key_iv: "44bf312d0879b7a4b1a0327266a51820"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.966387 29633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "ea20a457f9f044d299810b6861fe7a82"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "7d7c2dbe66404477eff384ec2eca69c0"
server_key_iv: "44bf312d0879b7a4b1a0327266a51820"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.971766 29633 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260808 23:40:58.975250 29647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:58.976688 29633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:40:58.976866 29633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "ea20a457f9f044d299810b6861fe7a82"
format_stamp: "Formatted at 2026-08-08 23:40:58 on dist-test-slave-5k54"
server_key: "7d7c2dbe66404477eff384ec2eca69c0"
server_key_iv: "44bf312d0879b7a4b1a0327266a51820"
server_key_version: "encryptionkey@0"
I20260808 23:40:58.977063 29633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:58.987638 29633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:58.988408 29633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:58.988708 29633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:58.989670 29633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:40:58.991142 29633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:40:58.991227 29633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:58.991382 29633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:40:58.991534 29633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 23:40:59.026552 29633 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:41171
I20260808 23:40:59.026611 29774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:41171 every 8 connection(s)
I20260808 23:40:59.028065 29633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:40:59.029930 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 29633
I20260808 23:40:59.030112 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:40:59.030639 19178 external_mini_cluster.cc:1428] Setting key 575607944c6a6e5dc5d9aec604e043ea
I20260808 23:40:59.040591 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:59.030730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:35379 (local address 127.18.186.190:46229)
0808 23:40:59.030960 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:59.030967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:59.032650 (+ 1683us) server_negotiation.cc:407] Connection header received
0808 23:40:59.034678 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:59.034681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:59.034744 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:59.034869 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:59.035966 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.036715 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.038172 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.038573 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.038668 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:59.039921 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:59.039926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:59.039929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:59.039946 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:59.039979 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:59.039990 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:59.040062 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:59.040265 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 23:40:59.040345 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:40:59.041525 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:59.031243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:46229 (local address 127.18.186.131:35379)
0808 23:40:59.032463 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0808 23:40:59.032513 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:40:59.034437 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:40:59.035060 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:40:59.035068 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:40:59.035154 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:40:59.035801 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:59.035817 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.036837 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.036840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:59.038037 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:40:59.038048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.038754 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.038759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:40:59.038949 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:59.039629 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:40:59.039632 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:40:59.039668 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:40:59.039742 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:40:59.040079 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:40:59.040086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:40:59.040091 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:40:59.040219 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 23:40:59.040501 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":401,"threads_started":1}
I20260808 23:40:59.043073 29775 heartbeater.cc:347] Connected to a master server at 127.18.186.190:46229
I20260808 23:40:59.043371 29775 heartbeater.cc:464] Registering TS with master...
I20260808 23:40:59.044144 29775 heartbeater.cc:511] Master 127.18.186.190:46229 requested a full tablet report, sending...
I20260808 23:40:59.045712 29279 ts_manager.cc:194] Registered new tserver with Master: ea20a457f9f044d299810b6861fe7a82 (127.18.186.131:41171)
I20260808 23:40:59.046473 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:40:59.046684 29279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.131:35379
W20260808 23:40:59.052860 29485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:40:59.051134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42520 (local address 127.18.186.190:46229)
0808 23:40:59.051316 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:59.051320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:59.051867 (+ 547us) server_negotiation.cc:407] Connection header received
0808 23:40:59.052062 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:59.052068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:59.052147 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:59.052252 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:59.052700 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42520: BlockingRecv error: recv got EOF from 127.0.0.1:42520 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:40:59.053763 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 29345
I20260808 23:40:59.061795 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:59.054736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33070 (local address 127.18.186.129:44667)
0808 23:40:59.055291 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:59.055297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:59.055322 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:40:59.055383 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:59.055389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:59.055657 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:59.055856 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:59.056250 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.057242 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.058537 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.059098 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.059243 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:59.060342 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:59.060351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:59.060362 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:59.060927 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:59.060972 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:59.061004 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:59.061147 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:59.061301 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 23:40:59.061437 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":307,"threads_started":1}
I20260808 23:40:59.062990 29478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:59.147282 29483 heartbeater.cc:503] Master 127.18.186.190:46229 was elected leader, sending a full tablet report...
I20260808 23:40:59.148525 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 29345
W20260808 23:40:59.166860 29261 connection.cc:535] server connection from 127.18.186.129:34509 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 23:40:59.167215 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 29487
I20260808 23:40:59.175504 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:59.168160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.18.186.130:42367)
0808 23:40:59.168712 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:59.168719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:59.168744 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:40:59.168810 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:59.168816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:59.169030 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:59.169205 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:59.169634 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.170671 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.172184 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.172763 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.172915 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:59.173958 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:59.173966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:59.173978 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:59.174606 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:59.174652 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:59.174672 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:59.174835 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:59.174966 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 23:40:59.175126 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":275,"threads_started":1}
I20260808 23:40:59.176791 29624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:59.259333 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 29487
I20260808 23:40:59.278728 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 29633
I20260808 23:40:59.287274 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:40:59.279841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.18.186.131:41171)
0808 23:40:59.280602 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0808 23:40:59.280623 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 23:40:59.280653 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:40:59.280712 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:40:59.280718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:40:59.280975 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:40:59.281162 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:40:59.281554 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.282634 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.284049 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:40:59.284624 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:40:59.284733 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:40:59.285747 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:40:59.285755 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:40:59.285809 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:40:59.286380 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:40:59.286426 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:40:59.286445 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:40:59.286542 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 23:40:59.286691 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 23:40:59.286901 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":451,"threads_started":1}
I20260808 23:40:59.288563 29769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:59.370815 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 29633
I20260808 23:40:59.389405 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 29243
I20260808 23:40:59.390681 29313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:40:59.494728 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 29243
2026-08-08T23:40:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2468 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T23:40:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:40:59Z Disabled control of system clock
I20260808 23:40:59.578918 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42149
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 23:40:59.919381 29797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:40:59.919685 29797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:40:59.919881 29797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:40:59.927256 29797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:40:59.927372 29797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:40:59.927410 29797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:40:59.927578 29797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:40:59.927641 29797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:40:59.937011 29797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42149
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:40:59.938356 29797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:40:59.940174 29797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:40:59.951952 29802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:59.952059 29803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:40:59.952039 29805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:40:59.953127 29797 server_base.cc:1034] running on GCE node
I20260808 23:40:59.953953 29797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:40:59.955272 29797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:40:59.956494 29797 hybrid_clock.cc:648] HybridClock initialized: now 1786232459956468 us; error 50 us; skew 500 ppm
I20260808 23:40:59.956898 29797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:40:59.959110 29797 webserver.cc:460] Webserver started at http://127.18.186.190:40243/ using document root <none> and password file <none>
I20260808 23:40:59.959889 29797 fs_manager.cc:359] Metadata directory not provided
I20260808 23:40:59.960019 29797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:40:59.960368 29797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:40:59.963301 29797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "5e61348019414a598efdcb9b083f2bfa"
format_stamp: "Formatted at 2026-08-08 23:40:59 on dist-test-slave-5k54"
server_key: "ce234b82e0064afcc0eac6b72c752b4b"
server_key_iv: "4979bae9e9b3e073ad5ec79bc983d7eb"
server_key_version: "encryptionkey@0"
I20260808 23:40:59.964166 29797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "5e61348019414a598efdcb9b083f2bfa"
format_stamp: "Formatted at 2026-08-08 23:40:59 on dist-test-slave-5k54"
server_key: "ce234b82e0064afcc0eac6b72c752b4b"
server_key_iv: "4979bae9e9b3e073ad5ec79bc983d7eb"
server_key_version: "encryptionkey@0"
I20260808 23:40:59.969202 29797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:40:59.972620 29811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:40:59.973897 29797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260808 23:40:59.974085 29797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "5e61348019414a598efdcb9b083f2bfa"
format_stamp: "Formatted at 2026-08-08 23:40:59 on dist-test-slave-5k54"
server_key: "ce234b82e0064afcc0eac6b72c752b4b"
server_key_iv: "4979bae9e9b3e073ad5ec79bc983d7eb"
server_key_version: "encryptionkey@0"
I20260808 23:40:59.974313 29797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:40:59.980129 29797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:40:59.980877 29797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:40:59.981180 29797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:40:59.999912 29797 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42149
I20260808 23:40:59.999998 29866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42149 every 8 connection(s)
I20260808 23:41:00.001302 29797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:41:00.002965 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 29797
I20260808 23:41:00.003199 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:41:00.004062 19178 external_mini_cluster.cc:1428] Setting key e40961a8ca2c60d6eac0ec9d065f0161
I20260808 23:41:00.006388 29867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:00.013403 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:00.006533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.18.186.190:42149)
0808 23:41:00.007680 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:00.007692 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:00.007739 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:41:00.009392 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:00.009417 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:00.009871 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:00.010351 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:00.011574 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:00.011581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:00.011594 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:41:00.011609 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:00.011692 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:41:00.011709 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:00.011977 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:00.012136 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 23:41:00.012408 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":390,"threads_started":1}
I20260808 23:41:00.019106 29867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa: Bootstrap starting.
I20260808 23:41:00.022630 29867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:00.023757 29867 log.cc:824] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:00.026515 29867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa: No bootstrap required, opened a new log
I20260808 23:41:00.033882 29867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e61348019414a598efdcb9b083f2bfa" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42149 } }
I20260808 23:41:00.034273 29867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:00.034373 29867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e61348019414a598efdcb9b083f2bfa, State: Initialized, Role: FOLLOWER
I20260808 23:41:00.034984 29867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e61348019414a598efdcb9b083f2bfa" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42149 } }
I20260808 23:41:00.035228 29867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:41:00.035392 29867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:41:00.035629 29867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:00.037886 29867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e61348019414a598efdcb9b083f2bfa" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42149 } }
I20260808 23:41:00.038416 29867 leader_election.cc:302] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e61348019414a598efdcb9b083f2bfa; no voters:
I20260808 23:41:00.039125 29867 leader_election.cc:288] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:41:00.039244 29872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:41:00.040050 29872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [term 1 LEADER]: Becoming Leader. State: Replica: 5e61348019414a598efdcb9b083f2bfa, State: Running, Role: LEADER
I20260808 23:41:00.040719 29872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e61348019414a598efdcb9b083f2bfa" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42149 } }
I20260808 23:41:00.041589 29867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:41:00.043707 29875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e61348019414a598efdcb9b083f2bfa. Latest consensus state: current_term: 1 leader_uuid: "5e61348019414a598efdcb9b083f2bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e61348019414a598efdcb9b083f2bfa" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42149 } } }
I20260808 23:41:00.044268 29874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e61348019414a598efdcb9b083f2bfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e61348019414a598efdcb9b083f2bfa" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42149 } } }
I20260808 23:41:00.044610 29875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [sys.catalog]: This master's current role is: LEADER
I20260808 23:41:00.044705 29874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa [sys.catalog]: This master's current role is: LEADER
I20260808 23:41:00.047533 29878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:41:00.052399 29878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:41:00.060966 29878 catalog_manager.cc:1347] Generated new cluster ID: 276f329c2ffe4518a53138e2257d22aa
I20260808 23:41:00.061151 29878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:41:00.078526 29878 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:41:00.079070 29878 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:41:00.088660 29878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e61348019414a598efdcb9b083f2bfa: Generated new TSK 0
I20260808 23:41:00.089460 29878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:41:00.099812 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42149
--builtin_ntp_servers=127.18.186.148:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 23:41:00.437433 29892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:00.437706 29892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:00.437870 29892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:00.445760 29892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:00.445873 29892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:00.446056 29892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:41:00.455464 29892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:00.456763 29892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:00.458599 29892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:00.471944 29901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:00.471925 29898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:00.472221 29899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:00.473233 29892 server_base.cc:1034] running on GCE node
I20260808 23:41:00.473861 29892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:00.474794 29892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:00.476025 29892 hybrid_clock.cc:648] HybridClock initialized: now 1786232460475978 us; error 66 us; skew 500 ppm
I20260808 23:41:00.476346 29892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:41:00.478873 29892 webserver.cc:460] Webserver started at http://127.18.186.129:37071/ using document root <none> and password file <none>
I20260808 23:41:00.479645 29892 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:00.479764 29892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:00.480110 29892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:41:00.482935 29892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "30a9b1ceb83f43668029e624fcf93ce0"
format_stamp: "Formatted at 2026-08-08 23:41:00 on dist-test-slave-5k54"
server_key: "4e6d705ca4991aebd1127dd53a21992a"
server_key_iv: "56bfa2746a7bc2fd62ec80065ed19fdc"
server_key_version: "encryptionkey@0"
I20260808 23:41:00.483973 29892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "30a9b1ceb83f43668029e624fcf93ce0"
format_stamp: "Formatted at 2026-08-08 23:41:00 on dist-test-slave-5k54"
server_key: "4e6d705ca4991aebd1127dd53a21992a"
server_key_iv: "56bfa2746a7bc2fd62ec80065ed19fdc"
server_key_version: "encryptionkey@0"
I20260808 23:41:00.489040 29892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:41:00.492568 29908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:00.493913 29892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:41:00.494086 29892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "30a9b1ceb83f43668029e624fcf93ce0"
format_stamp: "Formatted at 2026-08-08 23:41:00 on dist-test-slave-5k54"
server_key: "4e6d705ca4991aebd1127dd53a21992a"
server_key_iv: "56bfa2746a7bc2fd62ec80065ed19fdc"
server_key_version: "encryptionkey@0"
I20260808 23:41:00.494294 29892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:00.500034 29892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:00.500785 29892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:00.501056 29892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:00.502018 29892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:00.503471 29892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:00.503592 29892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:00.503698 29892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:00.503780 29892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:00.539008 29892 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:38367
I20260808 23:41:00.539104 30026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:38367 every 8 connection(s)
I20260808 23:41:00.540452 29892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:41:00.549106 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 29892
I20260808 23:41:00.549248 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:00.543323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:43735 (local address 127.18.186.190:42149)
0808 23:41:00.543866 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:00.543873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:00.545114 (+ 1241us) server_negotiation.cc:407] Connection header received
0808 23:41:00.546786 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:00.546793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:00.546867 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:00.546963 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:00.548479 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:00.548483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:00.548487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:41:00.548503 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:00.548548 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:41:00.548560 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:00.548657 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:00.548896 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 23:41:00.548967 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":271,"threads_started":1}
I20260808 23:41:00.549364 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:41:00.550019 19178 external_mini_cluster.cc:1428] Setting key 64475a768eb330c1fb3857ff100bb300
I20260808 23:41:00.550212 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:00.543999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42149 (local address 127.18.186.129:43735)
0808 23:41:00.544951 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:00.545001 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:00.546503 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:00.547228 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:00.547236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:00.547323 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:00.548143 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:41:00.548145 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:00.548211 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:41:00.548311 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:00.548652 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:00.548659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:00.548664 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:41:00.548933 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 23:41:00.549168 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":282,"threads_started":1}
I20260808 23:41:00.552167 30027 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42149
I20260808 23:41:00.552501 30027 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:00.553221 30027 heartbeater.cc:511] Master 127.18.186.190:42149 requested a full tablet report, sending...
I20260808 23:41:00.553363 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42149
--builtin_ntp_servers=127.18.186.148:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 23:41:00.555881 29829 ts_manager.cc:194] Registered new tserver with Master: 30a9b1ceb83f43668029e624fcf93ce0 (127.18.186.129:38367)
W20260808 23:41:00.887946 30031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:00.888213 30031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:00.888401 30031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:00.896203 30031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:00.896297 30031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:00.896411 30031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:41:00.905495 30031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:00.906688 30031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:00.908466 30031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:00.921370 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:00.921406 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:00.921437 30039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:00.922055 30031 server_base.cc:1034] running on GCE node
I20260808 23:41:00.922582 30031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:00.923413 30031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:00.924674 30031 hybrid_clock.cc:648] HybridClock initialized: now 1786232460924620 us; error 39 us; skew 500 ppm
I20260808 23:41:00.925035 30031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:41:00.927177 30031 webserver.cc:460] Webserver started at http://127.18.186.130:44665/ using document root <none> and password file <none>
I20260808 23:41:00.927966 30031 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:00.928083 30031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:00.928437 30031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:41:00.931247 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "aa3b365c4e914e2c866958eb79c346be"
format_stamp: "Formatted at 2026-08-08 23:41:00 on dist-test-slave-5k54"
server_key: "287b8355aad88775ee058453871dc8bb"
server_key_iv: "ba5f6f569f4b79ae2042f090e1cd643d"
server_key_version: "encryptionkey@0"
I20260808 23:41:00.932109 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "aa3b365c4e914e2c866958eb79c346be"
format_stamp: "Formatted at 2026-08-08 23:41:00 on dist-test-slave-5k54"
server_key: "287b8355aad88775ee058453871dc8bb"
server_key_iv: "ba5f6f569f4b79ae2042f090e1cd643d"
server_key_version: "encryptionkey@0"
I20260808 23:41:00.937068 30031 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:41:00.940546 30045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:00.941886 30031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:41:00.942065 30031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "aa3b365c4e914e2c866958eb79c346be"
format_stamp: "Formatted at 2026-08-08 23:41:00 on dist-test-slave-5k54"
server_key: "287b8355aad88775ee058453871dc8bb"
server_key_iv: "ba5f6f569f4b79ae2042f090e1cd643d"
server_key_version: "encryptionkey@0"
I20260808 23:41:00.942286 30031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:00.947942 30031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:00.948734 30031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:00.949028 30031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:00.949935 30031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:00.951278 30031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:00.951402 30031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:00.951593 30031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:00.951683 30031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:00.986030 30031 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:35409
I20260808 23:41:00.986124 30170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:35409 every 8 connection(s)
I20260808 23:41:00.987365 30031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:41:00.990516 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 30031
I20260808 23:41:00.990703 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:41:00.991263 19178 external_mini_cluster.cc:1428] Setting key 0251a97f80f2ad5fc42fae79ad37e291
I20260808 23:41:00.994211 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42149
--builtin_ntp_servers=127.18.186.148:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 23:41:00.997084 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:00.990194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:52049 (local address 127.18.186.190:42149)
0808 23:41:00.990412 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:00.990417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:00.992415 (+ 1998us) server_negotiation.cc:407] Connection header received
0808 23:41:00.994502 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:00.994507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:00.994574 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:00.994635 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:00.996256 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:00.996262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:00.996278 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:41:00.996302 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:00.996336 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:41:00.996367 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:00.996504 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:00.996713 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 23:41:00.996787 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 23:41:00.998104 30173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:00.990915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42149 (local address 127.18.186.130:52049)
0808 23:41:00.992246 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:00.992286 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:00.994147 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:00.995047 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:00.995056 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:00.995149 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:00.995928 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:41:00.995931 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:00.995978 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:41:00.996069 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:00.996476 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:00.996487 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:00.996495 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:41:00.996809 (+ 314us) client_negotiation.cc:240] Negotiation successful
0808 23:41:00.997062 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":401,"threads_started":1}
I20260808 23:41:00.999662 30171 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42149
I20260808 23:41:01.000041 30171 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:01.000780 30171 heartbeater.cc:511] Master 127.18.186.190:42149 requested a full tablet report, sending...
I20260808 23:41:01.002432 29829 ts_manager.cc:194] Registered new tserver with Master: aa3b365c4e914e2c866958eb79c346be (127.18.186.130:35409)
W20260808 23:41:01.327381 30175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:01.327783 30175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:01.327939 30175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:01.335333 30175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:01.335486 30175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:01.335676 30175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:41:01.345070 30175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:01.346345 30175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:01.348328 30175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:01.361600 30181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:01.361651 30180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:01.361689 30183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:01.362571 30175 server_base.cc:1034] running on GCE node
I20260808 23:41:01.363178 30175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:01.364151 30175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:01.365391 30175 hybrid_clock.cc:648] HybridClock initialized: now 1786232461365358 us; error 60 us; skew 500 ppm
I20260808 23:41:01.365713 30175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:41:01.368201 30175 webserver.cc:460] Webserver started at http://127.18.186.131:39355/ using document root <none> and password file <none>
I20260808 23:41:01.368909 30175 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:01.369048 30175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:01.369374 30175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:41:01.372291 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "b12c05a9f5df4461b888001603a98967"
format_stamp: "Formatted at 2026-08-08 23:41:01 on dist-test-slave-5k54"
server_key: "79c2ae7ec9cc1553ae14e922d0a5de6b"
server_key_iv: "90bfdb38cd4d49a36cbf1f8758c6426f"
server_key_version: "encryptionkey@0"
I20260808 23:41:01.373159 30175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "b12c05a9f5df4461b888001603a98967"
format_stamp: "Formatted at 2026-08-08 23:41:01 on dist-test-slave-5k54"
server_key: "79c2ae7ec9cc1553ae14e922d0a5de6b"
server_key_iv: "90bfdb38cd4d49a36cbf1f8758c6426f"
server_key_version: "encryptionkey@0"
I20260808 23:41:01.378392 30175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:41:01.381755 30189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:01.383236 30175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 23:41:01.383416 30175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "b12c05a9f5df4461b888001603a98967"
format_stamp: "Formatted at 2026-08-08 23:41:01 on dist-test-slave-5k54"
server_key: "79c2ae7ec9cc1553ae14e922d0a5de6b"
server_key_iv: "90bfdb38cd4d49a36cbf1f8758c6426f"
server_key_version: "encryptionkey@0"
I20260808 23:41:01.383692 30175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:01.389281 30175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:01.390034 30175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:01.390332 30175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:01.391230 30175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:01.392688 30175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:01.392817 30175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:01.392953 30175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:01.393029 30175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:01.427132 30175 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:36317
I20260808 23:41:01.427227 30313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:36317 every 8 connection(s)
I20260808 23:41:01.428591 30175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:41:01.433914 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 30175
I20260808 23:41:01.434129 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:41:01.434567 19178 external_mini_cluster.cc:1428] Setting key 53e88454e3e63f79843ec308fa8ff441
I20260808 23:41:01.437160 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:01.431205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:53031 (local address 127.18.186.190:42149)
0808 23:41:01.431400 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:01.431404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:01.432880 (+ 1476us) server_negotiation.cc:407] Connection header received
0808 23:41:01.434716 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:01.434722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:01.434804 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:01.434922 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:01.436369 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:01.436374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:01.436379 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:41:01.436397 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:01.436438 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:41:01.436449 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:01.436590 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:01.436784 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 23:41:01.436856 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:41:01.438076 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:01.431685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42149 (local address 127.18.186.131:53031)
0808 23:41:01.432663 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:01.432706 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:01.434446 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:01.435146 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:01.435153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:01.435266 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:01.436066 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:41:01.436068 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:01.436105 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:41:01.436188 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:01.436550 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:01.436563 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:01.436572 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 23:41:01.436878 (+ 306us) client_negotiation.cc:240] Negotiation successful
0808 23:41:01.437126 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":305,"threads_started":1}
I20260808 23:41:01.439455 30314 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42149
I20260808 23:41:01.439740 30314 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:01.440374 30314 heartbeater.cc:511] Master 127.18.186.190:42149 requested a full tablet report, sending...
I20260808 23:41:01.441682 29829 ts_manager.cc:194] Registered new tserver with Master: b12c05a9f5df4461b888001603a98967 (127.18.186.131:36317)
I20260808 23:41:01.450002 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 23:41:01.456489 30029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:41:01.454774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.18.186.190:42149)
0808 23:41:01.454951 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:01.454955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:01.455540 (+ 585us) server_negotiation.cc:407] Connection header received
0808 23:41:01.455685 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:01.455689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:01.455744 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:01.455867 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:01.456194 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47032: BlockingRecv error: recv got EOF from 127.0.0.1:47032 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:41:01.457006 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 29892
I20260808 23:41:01.460841 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:01.458125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.18.186.129:38367)
0808 23:41:01.458663 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:01.458670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:01.458694 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:41:01.458750 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:01.458755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:01.458964 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:01.459082 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:01.460119 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:01.460125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:01.460136 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:41:01.460155 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:01.460198 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:41:01.460212 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:01.460316 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:01.460438 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 23:41:01.460549 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":289,"threads_started":1}
I20260808 23:41:01.462085 30022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:01.545657 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 29892
I20260808 23:41:01.565409 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 30031
I20260808 23:41:01.569123 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:01.566337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.18.186.130:35409)
0808 23:41:01.566860 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:01.566867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:01.566892 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:41:01.566953 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:01.566959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:01.567213 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:01.567337 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:01.568404 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:01.568411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:01.568423 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:41:01.568443 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:01.568490 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:41:01.568505 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:01.568611 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:01.568697 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 23:41:01.568807 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":255,"threads_started":1}
I20260808 23:41:01.570374 30166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:01.652179 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 30031
I20260808 23:41:01.670574 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 30175
I20260808 23:41:01.674562 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:01.671657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.18.186.131:36317)
0808 23:41:01.672359 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:01.672366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:01.672396 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:41:01.672459 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:01.672464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:01.672679 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:01.672803 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:01.673723 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:01.673729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:01.673740 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:41:01.673756 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:01.673795 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:41:01.673808 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:01.673981 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:01.674106 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 23:41:01.674219 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":367,"threads_started":1}
I20260808 23:41:01.675932 30309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:01.757547 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 30175
I20260808 23:41:01.776633 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 29797
I20260808 23:41:01.777976 29862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:01.880528 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 29797
2026-08-08T23:41:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2386 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:41:01 dist-test-slave-5k54 krb5kdc[30333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:41:01 dist-test-slave-5k54 krb5kdc[30333](info): set up 2 sockets
Aug 08 23:41:01 dist-test-slave-5k54 krb5kdc[30333](info): commencing operation
krb5kdc: starting...
W20260808 23:41:03.046939 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:41:03 dist-test-slave-5k54 krb5kdc[30333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:41:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:41:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:41:03.312038 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:38091
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:38091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:41:03.643718 30349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:03.644068 30349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:03.644233 30349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:03.651607 30349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:41:03.651754 30349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:03.651862 30349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:03.651976 30349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:41:03.652061 30349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:41:03.661883 30349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:38091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:38091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:03.663285 30349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:03.664784 30349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:03.676776 30355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:03.676806 30358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:03.676862 30356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:03.678012 30349 server_base.cc:1034] running on GCE node
I20260808 23:41:03.678784 30349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:03.680280 30349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:03.681546 30349 hybrid_clock.cc:648] HybridClock initialized: now 1786232463681493 us; error 38 us; skew 500 ppm
I20260808 23:41:03.681916 30349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:41:03 dist-test-slave-5k54 krb5kdc[30333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232463, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:41:03.688091 30349 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:41:03.689620 30349 webserver.cc:460] Webserver started at http://127.18.186.190:32787/ using document root <none> and password file <none>
I20260808 23:41:03.690296 30349 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:03.690431 30349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:03.690757 30349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:41:03.693770 30349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "0d684f00e54940fabc61eb0720ed62b2"
format_stamp: "Formatted at 2026-08-08 23:41:03 on dist-test-slave-5k54"
server_key: "e319d9634ec7adb1dd8f41ebe55af01b"
server_key_iv: "d91618a439b2b2586f62430eb0035cb3"
server_key_version: "encryptionkey@0"
I20260808 23:41:03.694588 30349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "0d684f00e54940fabc61eb0720ed62b2"
format_stamp: "Formatted at 2026-08-08 23:41:03 on dist-test-slave-5k54"
server_key: "e319d9634ec7adb1dd8f41ebe55af01b"
server_key_iv: "d91618a439b2b2586f62430eb0035cb3"
server_key_version: "encryptionkey@0"
I20260808 23:41:03.699767 30349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 23:41:03.703022 30366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:03.704361 30349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:41:03.704536 30349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "0d684f00e54940fabc61eb0720ed62b2"
format_stamp: "Formatted at 2026-08-08 23:41:03 on dist-test-slave-5k54"
server_key: "e319d9634ec7adb1dd8f41ebe55af01b"
server_key_iv: "d91618a439b2b2586f62430eb0035cb3"
server_key_version: "encryptionkey@0"
I20260808 23:41:03.704722 30349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:03.728734 30349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:03.732949 30349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:03.733251 30349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:03.751643 30349 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:38091
I20260808 23:41:03.751679 30421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:38091 every 8 connection(s)
I20260808 23:41:03.753036 30349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:41:03.755360 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 30349
I20260808 23:41:03.755637 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:41:03.756474 19178 external_mini_cluster.cc:1428] Setting key c933f34964ed879bf7a56bc1cf70da31
I20260808 23:41:03.757347 30422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:03.769922 30422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2: Bootstrap starting.
Aug 08 23:41:03 dist-test-slave-5k54 krb5kdc[30333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:41:03.773650 30422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:03.774716 30422 log.cc:824] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:03.777529 30422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2: No bootstrap required, opened a new log
I20260808 23:41:03.778744 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:03.758598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.18.186.190:38091)
0808 23:41:03.759360 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:03.759383 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:03.759466 (+ 83us) server_negotiation.cc:407] Connection header received
0808 23:41:03.760975 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:03.760995 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:03.761376 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:03.761783 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:03.763315 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:03.764673 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:03.765946 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:03.766634 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:03.766847 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:03.771721 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:03.771733 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:03.771747 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:03.771787 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:03.774839 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:03.775568 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:03.775583 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:03.775590 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:03.775762 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:03.776169 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:03.776175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:03.776178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:03.776658 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:03.777037 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:03.777402 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 23:41:03.777819 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":346,"threads_started":1}
I20260808 23:41:03.786327 30422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d684f00e54940fabc61eb0720ed62b2" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38091 } }
I20260808 23:41:03.786689 30422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:03.786803 30422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d684f00e54940fabc61eb0720ed62b2, State: Initialized, Role: FOLLOWER
I20260808 23:41:03.787336 30422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d684f00e54940fabc61eb0720ed62b2" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38091 } }
I20260808 23:41:03.787642 30422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:41:03.787788 30422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:41:03.787983 30422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:03.790114 30422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d684f00e54940fabc61eb0720ed62b2" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38091 } }
I20260808 23:41:03.790591 30422 leader_election.cc:302] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d684f00e54940fabc61eb0720ed62b2; no voters:
I20260808 23:41:03.791198 30422 leader_election.cc:288] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:41:03.791360 30429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:41:03.792203 30429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [term 1 LEADER]: Becoming Leader. State: Replica: 0d684f00e54940fabc61eb0720ed62b2, State: Running, Role: LEADER
I20260808 23:41:03.792768 30429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d684f00e54940fabc61eb0720ed62b2" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38091 } }
I20260808 23:41:03.793359 30422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:41:03.795553 30431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d684f00e54940fabc61eb0720ed62b2. Latest consensus state: current_term: 1 leader_uuid: "0d684f00e54940fabc61eb0720ed62b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d684f00e54940fabc61eb0720ed62b2" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38091 } } }
I20260808 23:41:03.795948 30431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:41:03.801185 30430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d684f00e54940fabc61eb0720ed62b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d684f00e54940fabc61eb0720ed62b2" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38091 } } }
I20260808 23:41:03.801586 30430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:41:03.803275 30436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:41:03.808699 30436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:41:03.817431 30436 catalog_manager.cc:1347] Generated new cluster ID: bc0b09b755fc4ea9a3edc1cfe7efbde9
I20260808 23:41:03.817556 30436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:41:03.837788 30436 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:41:03.839685 30436 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:41:03.860971 30436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2: Generated new TSK 0
I20260808 23:41:03.862053 30436 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:41:03.934901 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38091
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:41:04.265823 30451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:04.266113 30451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:04.266273 30451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:04.273746 30451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:04.273862 30451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:04.274029 30451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:41:04.284561 30451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:04.285801 30451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:04.287082 30451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:04.300086 30459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:04.300136 30457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:04.300415 30456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:04.301034 30451 server_base.cc:1034] running on GCE node
I20260808 23:41:04.301645 30451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:04.302536 30451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:04.303735 30451 hybrid_clock.cc:648] HybridClock initialized: now 1786232464303674 us; error 53 us; skew 500 ppm
I20260808 23:41:04.304070 30451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:41:04 dist-test-slave-5k54 krb5kdc[30333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232464, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:41:04.307183 30451 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:41:04.308739 30451 webserver.cc:460] Webserver started at http://127.18.186.129:46103/ using document root <none> and password file <none>
I20260808 23:41:04.309530 30451 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:04.309681 30451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:04.309988 30451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:41:04.312885 30451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "fa629fcb60df4b509bcd9ced8a309bbe"
format_stamp: "Formatted at 2026-08-08 23:41:04 on dist-test-slave-5k54"
server_key: "6f0baa3dc538e12720c1e0c6a1391f43"
server_key_iv: "7ac96b7c75b84d7f658042acba93bac1"
server_key_version: "encryptionkey@0"
I20260808 23:41:04.313688 30451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "fa629fcb60df4b509bcd9ced8a309bbe"
format_stamp: "Formatted at 2026-08-08 23:41:04 on dist-test-slave-5k54"
server_key: "6f0baa3dc538e12720c1e0c6a1391f43"
server_key_iv: "7ac96b7c75b84d7f658042acba93bac1"
server_key_version: "encryptionkey@0"
I20260808 23:41:04.318773 30451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:41:04.322149 30467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:04.323542 30451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:41:04.323716 30451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "fa629fcb60df4b509bcd9ced8a309bbe"
format_stamp: "Formatted at 2026-08-08 23:41:04 on dist-test-slave-5k54"
server_key: "6f0baa3dc538e12720c1e0c6a1391f43"
server_key_iv: "7ac96b7c75b84d7f658042acba93bac1"
server_key_version: "encryptionkey@0"
I20260808 23:41:04.323927 30451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:04.343819 30451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:04.348299 30451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:04.348610 30451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:04.349548 30451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:04.350886 30451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:04.351001 30451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:04.351173 30451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:04.351248 30451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:04.384838 30451 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:39031
I20260808 23:41:04.384886 30593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:39031 every 8 connection(s)
I20260808 23:41:04.386126 30451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:41:04.392181 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 30451
I20260808 23:41:04.392448 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:41:04.393151 19178 external_mini_cluster.cc:1428] Setting key 45218017ef12cb0d0aebcaec8b133569
Aug 08 23:41:04 dist-test-slave-5k54 krb5kdc[30333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232464, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:41:04.406265 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:04.388726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:34359 (local address 127.18.186.190:38091)
0808 23:41:04.389226 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:04.389232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:04.390211 (+ 979us) server_negotiation.cc:407] Connection header received
0808 23:41:04.392197 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:04.392201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:04.392267 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:04.392393 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:04.394029 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:04.394857 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:04.396480 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:04.396851 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:04.396996 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:04.400415 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:04.400428 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:04.400432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:04.400463 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:04.402993 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:04.403859 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:04.403874 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:04.403877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:04.403993 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:04.404428 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:04.404434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:04.404437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:04.404807 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:04.405012 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:04.405680 (+ 668us) server_negotiation.cc:299] Negotiation successful
0808 23:41:04.405911 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":237,"threads_started":1}
I20260808 23:41:04.407207 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:04.389349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:38091 (local address 127.18.186.129:34359)
0808 23:41:04.390043 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:04.390086 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:04.391846 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:04.392672 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:04.392680 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:04.393199 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:04.393852 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:04.393870 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:04.395012 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:04.395022 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:04.396318 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:04.396330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:04.397084 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:04.397094 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:04.397333 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:04.398006 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:41:04.398032 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:04.400203 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:04.403139 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:04.403147 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:04.403162 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:04.403733 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:04.404108 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:04.404112 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:04.404116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:04.404303 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:04.404965 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:04.404973 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:04.405306 (+ 333us) client_negotiation.cc:769] Sending connection context
0808 23:41:04.405568 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 23:41:04.406071 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":258,"threads_started":1}
I20260808 23:41:04.408895 30594 heartbeater.cc:347] Connected to a master server at 127.18.186.190:38091
I20260808 23:41:04.409192 30594 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:04.409919 30594 heartbeater.cc:511] Master 127.18.186.190:38091 requested a full tablet report, sending...
I20260808 23:41:04.411965 30383 ts_manager.cc:194] Registered new tserver with Master: fa629fcb60df4b509bcd9ced8a309bbe (127.18.186.129:39031)
I20260808 23:41:04.413369 30383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:34359
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:41:04.452634 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38091
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:41:04.793083 30602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:04.793378 30602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:04.793578 30602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:04.800936 30602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:04.801090 30602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:04.801276 30602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:41:04.811690 30602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:04.813004 30602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:04.814350 30602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:04.827695 30609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:04.827651 30610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:04.827673 30612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:04.828842 30602 server_base.cc:1034] running on GCE node
I20260808 23:41:04.829486 30602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:04.830392 30602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:04.831611 30602 hybrid_clock.cc:648] HybridClock initialized: now 1786232464831565 us; error 53 us; skew 500 ppm
I20260808 23:41:04.831956 30602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:41:04 dist-test-slave-5k54 krb5kdc[30333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232464, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:41:04.835259 30602 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:41:04.837059 30602 webserver.cc:460] Webserver started at http://127.18.186.130:36433/ using document root <none> and password file <none>
I20260808 23:41:04.837781 30602 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:04.837932 30602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:04.838274 30602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:41:04.841316 30602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "d9dd5b7ed2204f90bebca1481a748a39"
format_stamp: "Formatted at 2026-08-08 23:41:04 on dist-test-slave-5k54"
server_key: "3fab7b2c129e60ee5028051750742a39"
server_key_iv: "609ff3ffd4cd0d6c33ce4528fae89db0"
server_key_version: "encryptionkey@0"
I20260808 23:41:04.842131 30602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "d9dd5b7ed2204f90bebca1481a748a39"
format_stamp: "Formatted at 2026-08-08 23:41:04 on dist-test-slave-5k54"
server_key: "3fab7b2c129e60ee5028051750742a39"
server_key_iv: "609ff3ffd4cd0d6c33ce4528fae89db0"
server_key_version: "encryptionkey@0"
I20260808 23:41:04.847402 30602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 23:41:04.850811 30619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:04.852186 30602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:41:04.852357 30602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "d9dd5b7ed2204f90bebca1481a748a39"
format_stamp: "Formatted at 2026-08-08 23:41:04 on dist-test-slave-5k54"
server_key: "3fab7b2c129e60ee5028051750742a39"
server_key_iv: "609ff3ffd4cd0d6c33ce4528fae89db0"
server_key_version: "encryptionkey@0"
I20260808 23:41:04.852576 30602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:04.875766 30602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:04.880131 30602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:04.880393 30602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:04.881313 30602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:04.882663 30602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:04.882740 30602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:04.882824 30602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:04.882869 30602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:04.916613 30602 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:44355
I20260808 23:41:04.916656 30740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:44355 every 8 connection(s)
I20260808 23:41:04.917893 30602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:41:04.918599 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 30602
I20260808 23:41:04.918833 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:41:04.919380 19178 external_mini_cluster.cc:1428] Setting key 1581510638b44ac47a022f3d7a5e0013
Aug 08 23:41:04 dist-test-slave-5k54 krb5kdc[30333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232464, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:41:04.938650 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:04.920514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:38227 (local address 127.18.186.190:38091)
0808 23:41:04.920970 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:04.920974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:04.922106 (+ 1132us) server_negotiation.cc:407] Connection header received
0808 23:41:04.924011 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:04.924017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:04.924106 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:04.924225 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:04.925697 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:04.926435 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:04.928165 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:04.928729 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:04.928863 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:04.932298 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:04.932308 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:04.932315 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:04.932361 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:04.935314 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:04.936262 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:04.936266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:04.936268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:04.936401 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:04.936902 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:04.936905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:04.936907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:04.937147 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:04.937265 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:04.938179 (+ 914us) server_negotiation.cc:299] Negotiation successful
0808 23:41:04.938367 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":255,"threads_started":1}
I20260808 23:41:04.940197 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:04.921171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:38091 (local address 127.18.186.130:38227)
0808 23:41:04.921966 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:04.922003 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:04.923755 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:04.924457 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:04.924465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:04.924970 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:04.925510 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:04.925527 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:04.926595 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:04.926599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:04.928029 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:04.928040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:04.928959 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:04.928968 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:04.929195 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:04.929883 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:41:04.929913 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:04.932077 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:04.935494 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:04.935505 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:04.935523 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:04.936138 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:04.936521 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:04.936528 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:04.936535 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:04.936786 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:04.937284 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:04.937294 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:04.937740 (+ 446us) client_negotiation.cc:769] Sending connection context
0808 23:41:04.938065 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 23:41:04.938650 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":324,"threads_started":1}
I20260808 23:41:04.941828 30741 heartbeater.cc:347] Connected to a master server at 127.18.186.190:38091
I20260808 23:41:04.942123 30741 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:04.942919 30741 heartbeater.cc:511] Master 127.18.186.190:38091 requested a full tablet report, sending...
I20260808 23:41:04.944538 30383 ts_manager.cc:194] Registered new tserver with Master: d9dd5b7ed2204f90bebca1481a748a39 (127.18.186.130:44355)
I20260808 23:41:04.945847 30383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:38227
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:41:04.982306 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38091
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:41:05.310390 30748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:05.310649 30748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:05.310743 30748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:05.317711 30748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:05.317808 30748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:05.317926 30748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:41:05.327108 30748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:05.328287 30748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:05.329514 30748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:05.342492 30756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:05.342561 30759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:05.342492 30757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:05.343685 30748 server_base.cc:1034] running on GCE node
I20260808 23:41:05.344326 30748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:05.345218 30748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:05.346410 30748 hybrid_clock.cc:648] HybridClock initialized: now 1786232465346357 us; error 45 us; skew 500 ppm
I20260808 23:41:05.346714 30748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:41:05 dist-test-slave-5k54 krb5kdc[30333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232465, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:41:05.349888 30748 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:41:05.351315 30748 webserver.cc:460] Webserver started at http://127.18.186.131:45291/ using document root <none> and password file <none>
I20260808 23:41:05.352035 30748 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:05.352190 30748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:05.352491 30748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:41:05.355284 30748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "fcbbb0005e0d4fb2939154c289c775fe"
format_stamp: "Formatted at 2026-08-08 23:41:05 on dist-test-slave-5k54"
server_key: "3ad5dd16b744d09174a5982b01333106"
server_key_iv: "3bd6bc590e5085acc9804d859d987d34"
server_key_version: "encryptionkey@0"
I20260808 23:41:05.356288 30748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "fcbbb0005e0d4fb2939154c289c775fe"
format_stamp: "Formatted at 2026-08-08 23:41:05 on dist-test-slave-5k54"
server_key: "3ad5dd16b744d09174a5982b01333106"
server_key_iv: "3bd6bc590e5085acc9804d859d987d34"
server_key_version: "encryptionkey@0"
I20260808 23:41:05.361380 30748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:41:05.364653 30766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:05.365887 30748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:41:05.366063 30748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "fcbbb0005e0d4fb2939154c289c775fe"
format_stamp: "Formatted at 2026-08-08 23:41:05 on dist-test-slave-5k54"
server_key: "3ad5dd16b744d09174a5982b01333106"
server_key_iv: "3bd6bc590e5085acc9804d859d987d34"
server_key_version: "encryptionkey@0"
I20260808 23:41:05.366250 30748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:05.377858 30748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:05.381803 30748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:05.382118 30748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:05.382999 30748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:05.384441 30748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:05.384537 30748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:05.384658 30748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:05.384721 30748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:05.416357 30594 heartbeater.cc:503] Master 127.18.186.190:38091 was elected leader, sending a full tablet report...
I20260808 23:41:05.419298 30748 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:34879
I20260808 23:41:05.419335 30888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:34879 every 8 connection(s)
I20260808 23:41:05.420581 30748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:41:05.426944 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 30748
I20260808 23:41:05.427152 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:41:05.427695 19178 external_mini_cluster.cc:1428] Setting key 10fff73c9d6efabb5e8fb2012b191b2c
Aug 08 23:41:05 dist-test-slave-5k54 krb5kdc[30333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232465, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:41:05.440209 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.423059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:48167 (local address 127.18.186.190:38091)
0808 23:41:05.423228 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.423232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.424464 (+ 1232us) server_negotiation.cc:407] Connection header received
0808 23:41:05.426288 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.426294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.426398 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.426512 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:05.428503 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.429597 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.431259 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.431690 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.431792 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.435088 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:05.435093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:05.435096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:05.435129 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:05.437602 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:05.438267 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:05.438271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:05.438273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:05.438395 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:05.438761 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:05.438764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:05.438766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:05.439035 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:05.439145 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.439802 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.439976 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:41:05.441337 30891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.423606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:38091 (local address 127.18.186.131:48167)
0808 23:41:05.424298 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:05.424341 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:05.426023 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:05.426786 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:05.426795 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:05.427485 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:05.428253 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.428281 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.429742 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.429746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.431135 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.431147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.431873 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.431880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.432071 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.432672 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:41:05.432694 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:05.434880 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:05.437739 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:05.437745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:05.437757 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:05.438160 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:05.438488 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:05.438491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:05.438493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:05.438671 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:05.439138 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:05.439143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:05.439482 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 23:41:05.439717 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 23:41:05.440100 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":271,"threads_started":1}
I20260808 23:41:05.442688 30889 heartbeater.cc:347] Connected to a master server at 127.18.186.190:38091
I20260808 23:41:05.442986 30889 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:05.443754 30889 heartbeater.cc:511] Master 127.18.186.190:38091 requested a full tablet report, sending...
I20260808 23:41:05.445098 30383 ts_manager.cc:194] Registered new tserver with Master: fcbbb0005e0d4fb2939154c289c775fe (127.18.186.131:34879)
I20260808 23:41:05.445892 30383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:48167
I20260808 23:41:05.455132 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:41:05.472904 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.459744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59352 (local address 127.18.186.190:38091)
0808 23:41:05.459935 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.459940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.460453 (+ 513us) server_negotiation.cc:407] Connection header received
0808 23:41:05.460664 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.460668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.460728 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.460843 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:05.462264 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.463061 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.464346 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.464749 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.464911 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.467191 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:05.467196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:05.467199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:05.467230 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:05.469886 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:05.470557 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:05.470565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:05.470568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:05.470758 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:05.471325 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:05.471331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:05.471335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:05.471730 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:05.472024 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.472359 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.472578 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 23:41:05.476796 30383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:41:05.483590 30383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:41:05.521315 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.495952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54868 (local address 127.18.186.129:39031)
0808 23:41:05.511923 (+ 15971us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.511932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.511963 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:41:05.512071 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.512080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.513815 (+ 1735us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.514041 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:05.514891 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.516354 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.518679 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.519931 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.520561 (+ 630us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.520749 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.520837 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.520980 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15773,"thread_start_us":17438,"threads_started":1}
I20260808 23:41:05.521425 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.495650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39031 (local address 127.0.0.1:54868)
0808 23:41:05.496364 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:05.496408 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:05.496945 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:05.514056 (+ 17111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:05.514065 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:05.514093 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:05.514548 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.514562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.516513 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.516521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.518510 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.518522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.520210 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.520221 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.520528 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.520598 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 23:41:05.520718 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 23:41:05.520908 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":1,"thread_start_us":301,"threads_started":1}
I20260808 23:41:05.524281 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.496511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:34879 (local address 127.0.0.1:52302)
0808 23:41:05.508508 (+ 11997us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:05.508534 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:05.508663 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:05.516596 (+ 7933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:05.516603 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:05.516633 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:05.517256 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.517268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.519207 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.519211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.521504 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.521521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.523396 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.523402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.523627 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.523658 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:41:05.523710 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 23:41:05.523837 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11862,"mutex_wait_us":33,"thread_start_us":11332,"threads_started":1}
I20260808 23:41:05.524417 30908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.497438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.18.186.130:44355)
0808 23:41:05.515845 (+ 18407us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.515852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.515879 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:41:05.515947 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.515952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.516092 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.516284 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:05.517146 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.518723 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.521837 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.523193 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.523362 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.523543 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.523910 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.524096 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18258,"thread_start_us":18120,"threads_started":1}
I20260808 23:41:05.524344 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.496770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:44355 (local address 127.0.0.1:40524)
0808 23:41:05.509608 (+ 12838us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:05.509631 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:05.509769 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:05.516269 (+ 6500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:05.516273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:05.516298 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:05.516958 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.516973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.518888 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.518894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.521669 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.521682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.523468 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.523476 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.523677 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.523708 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:41:05.523806 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 23:41:05.523922 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12688,"mutex_wait_us":392,"thread_start_us":11143,"threads_started":1}
I20260808 23:41:05.525559 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.497364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.18.186.131:34879)
0808 23:41:05.514827 (+ 17463us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.514838 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.514885 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:41:05.514993 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.515003 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.515248 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.516617 (+ 1369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:05.517452 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.519027 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.521715 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.523152 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.524742 (+ 1590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.524925 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.525025 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.525187 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17270,"thread_start_us":17125,"threads_started":1}
I20260808 23:41:05.525653 30520 tablet_service.cc:1467] Processing CreateTablet for tablet 786a68410b774de9a8c2191ecfa96f90 (DEFAULT_TABLE table=test-table [id=1a0592e806c046f88baffd6b751c921a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:41:05.527201 30520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786a68410b774de9a8c2191ecfa96f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:05.528767 30819 tablet_service.cc:1467] Processing CreateTablet for tablet 786a68410b774de9a8c2191ecfa96f90 (DEFAULT_TABLE table=test-table [id=1a0592e806c046f88baffd6b751c921a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:41:05.528770 30668 tablet_service.cc:1467] Processing CreateTablet for tablet 786a68410b774de9a8c2191ecfa96f90 (DEFAULT_TABLE table=test-table [id=1a0592e806c046f88baffd6b751c921a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:41:05.530099 30668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786a68410b774de9a8c2191ecfa96f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:05.530097 30819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786a68410b774de9a8c2191ecfa96f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:05.539551 30910 tablet_bootstrap.cc:492] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Bootstrap starting.
I20260808 23:41:05.542404 30910 tablet_bootstrap.cc:654] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:05.543505 30910 log.cc:824] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:05.544983 30912 tablet_bootstrap.cc:492] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Bootstrap starting.
I20260808 23:41:05.546509 30913 tablet_bootstrap.cc:492] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Bootstrap starting.
I20260808 23:41:05.546864 30910 tablet_bootstrap.cc:492] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: No bootstrap required, opened a new log
I20260808 23:41:05.547140 30910 ts_tablet_manager.cc:1397] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 23:41:05.548648 30912 tablet_bootstrap.cc:654] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:05.549769 30913 tablet_bootstrap.cc:654] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:05.549924 30912 log.cc:824] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:05.550801 30913 log.cc:824] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:05.553058 30912 tablet_bootstrap.cc:492] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: No bootstrap required, opened a new log
I20260808 23:41:05.553318 30912 ts_tablet_manager.cc:1397] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 23:41:05.555280 30913 tablet_bootstrap.cc:492] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: No bootstrap required, opened a new log
I20260808 23:41:05.555577 30913 ts_tablet_manager.cc:1397] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.007s
I20260808 23:41:05.557900 30910 raft_consensus.cc:348] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.558327 30910 raft_consensus.cc:374] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:05.558421 30910 raft_consensus.cc:729] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa629fcb60df4b509bcd9ced8a309bbe, State: Initialized, Role: FOLLOWER
I20260808 23:41:05.559099 30910 consensus_queue.cc:260] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.560840 30910 ts_tablet_manager.cc:1428] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260808 23:41:05.563024 30912 raft_consensus.cc:348] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.563367 30912 raft_consensus.cc:374] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:05.563510 30912 raft_consensus.cc:729] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcbbb0005e0d4fb2939154c289c775fe, State: Initialized, Role: FOLLOWER
I20260808 23:41:05.564103 30912 consensus_queue.cc:260] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.565379 30889 heartbeater.cc:503] Master 127.18.186.190:38091 was elected leader, sending a full tablet report...
I20260808 23:41:05.565181 30913 raft_consensus.cc:348] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.565624 30913 raft_consensus.cc:374] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:05.565727 30913 raft_consensus.cc:729] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9dd5b7ed2204f90bebca1481a748a39, State: Initialized, Role: FOLLOWER
I20260808 23:41:05.566318 30913 consensus_queue.cc:260] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.567155 30912 ts_tablet_manager.cc:1428] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260808 23:41:05.567559 30741 heartbeater.cc:503] Master 127.18.186.190:38091 was elected leader, sending a full tablet report...
I20260808 23:41:05.568104 30913 ts_tablet_manager.cc:1428] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260808 23:41:05.603767 30918 raft_consensus.cc:482] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:41:05.604017 30918 raft_consensus.cc:504] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.604497 30917 raft_consensus.cc:482] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:41:05.604800 30917 raft_consensus.cc:504] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.606380 30918 leader_election.cc:288] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa629fcb60df4b509bcd9ced8a309bbe (127.18.186.129:39031), fcbbb0005e0d4fb2939154c289c775fe (127.18.186.131:34879)
I20260808 23:41:05.606401 30917 leader_election.cc:288] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa629fcb60df4b509bcd9ced8a309bbe (127.18.186.129:39031), d9dd5b7ed2204f90bebca1481a748a39 (127.18.186.130:44355)
I20260808 23:41:05.615442 30891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.606591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39031 (local address 127.18.186.131:50315)
0808 23:41:05.607282 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:05.607308 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:05.607485 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:05.608882 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:05.608887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:05.608910 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:05.609090 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.609102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.610600 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.610605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.612875 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.612890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.614645 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.614653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.614829 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.614879 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 23:41:05.614970 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 23:41:05.615112 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537}
I20260808 23:41:05.615680 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.607935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:36279 (local address 127.18.186.131:34879)
0808 23:41:05.608224 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.608232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.608300 (+ 68us) server_negotiation.cc:407] Connection header received
0808 23:41:05.608618 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.608623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.608696 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.608752 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:05.609647 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.610724 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.613294 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.614557 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.614666 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.614730 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.615309 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.615464 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260808 23:41:05.616312 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.606692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:50315 (local address 127.18.186.129:39031)
0808 23:41:05.606912 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.606917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.608444 (+ 1527us) server_negotiation.cc:407] Connection header received
0808 23:41:05.608523 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.608530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.608609 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.608692 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:05.609271 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.610395 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.613076 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.614329 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.614487 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.614565 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.615710 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.616043 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:41:05.616341 30925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.607020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:34879 (local address 127.18.186.130:36279)
0808 23:41:05.608200 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:05.608222 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:05.608422 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:05.608999 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:05.609005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:05.609034 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:05.609432 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.609443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.610884 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.610888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.613131 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.613144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.614942 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.614951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.615132 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.615163 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:41:05.615883 (+ 720us) client_negotiation.cc:240] Negotiation successful
0808 23:41:05.616024 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"threads_started":1}
I20260808 23:41:05.617678 30540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a68410b774de9a8c2191ecfa96f90" candidate_uuid: "fcbbb0005e0d4fb2939154c289c775fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" is_pre_election: true
I20260808 23:41:05.618317 30540 raft_consensus.cc:2454] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcbbb0005e0d4fb2939154c289c775fe in term 0.
I20260808 23:41:05.618386 30840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a68410b774de9a8c2191ecfa96f90" candidate_uuid: "d9dd5b7ed2204f90bebca1481a748a39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcbbb0005e0d4fb2939154c289c775fe" is_pre_election: true
I20260808 23:41:05.618789 30840 raft_consensus.cc:2454] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9dd5b7ed2204f90bebca1481a748a39 in term 0.
I20260808 23:41:05.619220 30768 leader_election.cc:302] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa629fcb60df4b509bcd9ced8a309bbe, fcbbb0005e0d4fb2939154c289c775fe; no voters:
I20260808 23:41:05.619546 30623 leader_election.cc:302] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9dd5b7ed2204f90bebca1481a748a39, fcbbb0005e0d4fb2939154c289c775fe; no voters:
I20260808 23:41:05.619715 30917 raft_consensus.cc:2781] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:41:05.619894 30917 raft_consensus.cc:482] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:41:05.619979 30918 raft_consensus.cc:2781] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:41:05.620020 30917 raft_consensus.cc:3037] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:05.620122 30918 raft_consensus.cc:482] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:41:05.620232 30918 raft_consensus.cc:3037] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:05.623546 30917 raft_consensus.cc:504] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.623567 30918 raft_consensus.cc:504] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.624230 30917 leader_election.cc:288] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [CANDIDATE]: Term 1 election: Requested vote from peers fa629fcb60df4b509bcd9ced8a309bbe (127.18.186.129:39031), d9dd5b7ed2204f90bebca1481a748a39 (127.18.186.130:44355)
I20260808 23:41:05.624512 30908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.608204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:56041 (local address 127.18.186.130:44355)
0808 23:41:05.608399 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.608405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.616599 (+ 8194us) server_negotiation.cc:407] Connection header received
0808 23:41:05.617130 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.617135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.617204 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.617280 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:05.618069 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.619151 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.621950 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.623295 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.623411 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.623495 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.624071 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.624205 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 23:41:05.624907 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.606938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:44355 (local address 127.18.186.131:56041)
0808 23:41:05.616481 (+ 9543us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:05.616503 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:05.616972 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:05.617478 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:05.617482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:05.617510 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:05.617881 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.617894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.619550 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.619556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.621755 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.621770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.623683 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.623692 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.623869 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.623906 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 23:41:05.624379 (+ 473us) client_negotiation.cc:240] Negotiation successful
0808 23:41:05.624523 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9404,"thread_start_us":9679,"threads_started":1}
I20260808 23:41:05.624893 30540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a68410b774de9a8c2191ecfa96f90" candidate_uuid: "fcbbb0005e0d4fb2939154c289c775fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa629fcb60df4b509bcd9ced8a309bbe"
I20260808 23:41:05.625089 30540 raft_consensus.cc:3037] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:05.625186 30840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a68410b774de9a8c2191ecfa96f90" candidate_uuid: "d9dd5b7ed2204f90bebca1481a748a39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcbbb0005e0d4fb2939154c289c775fe"
I20260808 23:41:05.625336 30925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.607312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39031 (local address 127.18.186.130:52505)
0808 23:41:05.616518 (+ 9206us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:05.616538 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:05.616641 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:05.617691 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:05.617696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:05.617719 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:05.617886 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.617895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.620228 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.620235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.622258 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:05.622271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.624529 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.624535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:05.624740 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.624792 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 23:41:05.624866 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 23:41:05.625003 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9108,"thread_start_us":4133,"threads_started":1}
I20260808 23:41:05.625453 30840 raft_consensus.cc:2379] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d9dd5b7ed2204f90bebca1481a748a39 in current term 1: Already voted for candidate fcbbb0005e0d4fb2939154c289c775fe in this term.
I20260808 23:41:05.626314 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.607809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:52505 (local address 127.18.186.129:39031)
0808 23:41:05.617064 (+ 9255us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.617070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.617094 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:41:05.617146 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.617150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.617219 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.618070 (+ 851us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:05.618369 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.619961 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.622521 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.624302 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.625658 (+ 1356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.625733 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.625786 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.625933 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9066,"thread_start_us":8666,"threads_started":1}
I20260808 23:41:05.624331 30918 leader_election.cc:288] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [CANDIDATE]: Term 1 election: Requested vote from peers fa629fcb60df4b509bcd9ced8a309bbe (127.18.186.129:39031), fcbbb0005e0d4fb2939154c289c775fe (127.18.186.131:34879)
I20260808 23:41:05.626614 30689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a68410b774de9a8c2191ecfa96f90" candidate_uuid: "fcbbb0005e0d4fb2939154c289c775fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9dd5b7ed2204f90bebca1481a748a39" is_pre_election: true
I20260808 23:41:05.627350 30689 raft_consensus.cc:2379] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fcbbb0005e0d4fb2939154c289c775fe in current term 1: Already voted for candidate d9dd5b7ed2204f90bebca1481a748a39 in this term.
I20260808 23:41:05.627524 30538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a68410b774de9a8c2191ecfa96f90" candidate_uuid: "d9dd5b7ed2204f90bebca1481a748a39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa629fcb60df4b509bcd9ced8a309bbe"
I20260808 23:41:05.626622 30688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a68410b774de9a8c2191ecfa96f90" candidate_uuid: "fcbbb0005e0d4fb2939154c289c775fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9dd5b7ed2204f90bebca1481a748a39"
I20260808 23:41:05.627965 30688 raft_consensus.cc:2379] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fcbbb0005e0d4fb2939154c289c775fe in current term 1: Already voted for candidate d9dd5b7ed2204f90bebca1481a748a39 in this term.
I20260808 23:41:05.628224 30539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a68410b774de9a8c2191ecfa96f90" candidate_uuid: "d9dd5b7ed2204f90bebca1481a748a39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" is_pre_election: true
I20260808 23:41:05.628631 30621 leader_election.cc:302] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9dd5b7ed2204f90bebca1481a748a39; no voters: fa629fcb60df4b509bcd9ced8a309bbe, fcbbb0005e0d4fb2939154c289c775fe
I20260808 23:41:05.628865 30540 raft_consensus.cc:2454] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcbbb0005e0d4fb2939154c289c775fe in term 1.
I20260808 23:41:05.628973 30918 raft_consensus.cc:2726] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 23:41:05.629252 30768 leader_election.cc:302] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: fa629fcb60df4b509bcd9ced8a309bbe, fcbbb0005e0d4fb2939154c289c775fe; no voters: d9dd5b7ed2204f90bebca1481a748a39
I20260808 23:41:05.629526 30917 raft_consensus.cc:2781] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:41:05.630010 30917 raft_consensus.cc:686] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 1 LEADER]: Becoming Leader. State: Replica: fcbbb0005e0d4fb2939154c289c775fe, State: Running, Role: LEADER
I20260808 23:41:05.630721 30917 consensus_queue.cc:237] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } }
I20260808 23:41:05.635339 30383 catalog_manager.cc:5495] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe reported cstate change: term changed from 0 to 1, leader changed from <none> to fcbbb0005e0d4fb2939154c289c775fe (127.18.186.131). New cstate: current_term: 1 leader_uuid: "fcbbb0005e0d4fb2939154c289c775fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcbbb0005e0d4fb2939154c289c775fe" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 34879 } health_report { overall_health: HEALTHY } } }
W20260808 23:41:05.638451 30595 tablet.cc:2369] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:41:05.669902 30743 tablet.cc:2369] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:41:05.672202 30890 tablet.cc:2369] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:41:05.674095 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.668329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.18.186.131:34879)
0808 23:41:05.668547 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.668556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.668646 (+ 90us) server_negotiation.cc:407] Connection header received
0808 23:41:05.668822 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.668826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.668929 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.669091 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:41:05.669389 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.670290 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.671811 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.672462 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.672597 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.673034 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:41:05.673511 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:41:05.673626 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.673700 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.673766 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:41:05.682070 30540 raft_consensus.cc:1261] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 1 FOLLOWER]: Refusing update from remote peer fcbbb0005e0d4fb2939154c289c775fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:41:05.682130 30688 raft_consensus.cc:1261] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 1 FOLLOWER]: Refusing update from remote peer fcbbb0005e0d4fb2939154c289c775fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:41:05.682966 30927 consensus_queue.cc:1035] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa629fcb60df4b509bcd9ced8a309bbe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:41:05.683362 30917 consensus_queue.cc:1035] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9dd5b7ed2204f90bebca1481a748a39" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:41:05.709635 30383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:41:05.709926 30383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:41:05.713467 30383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d684f00e54940fabc61eb0720ed62b2: Sending DeleteTablet for 3 replicas of tablet 786a68410b774de9a8c2191ecfa96f90
I20260808 23:41:05.714562 30520 tablet_service.cc:1514] Processing DeleteTablet for tablet 786a68410b774de9a8c2191ecfa96f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:41:05 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:54868
I20260808 23:41:05.714579 30668 tablet_service.cc:1514] Processing DeleteTablet for tablet 786a68410b774de9a8c2191ecfa96f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:41:05 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:40524
I20260808 23:41:05.714586 30819 tablet_service.cc:1514] Processing DeleteTablet for tablet 786a68410b774de9a8c2191ecfa96f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:41:05 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:52302
I20260808 23:41:05.717482 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 30451
I20260808 23:41:05.727733 30941 tablet_replica.cc:330] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: stopping tablet replica
I20260808 23:41:05.728018 30942 tablet_replica.cc:330] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: stopping tablet replica
I20260808 23:41:05.728205 30941 raft_consensus.cc:2229] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:41:05.728349 30940 tablet_replica.cc:330] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: stopping tablet replica
I20260808 23:41:05.728633 30942 raft_consensus.cc:2229] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:41:05.728747 30941 raft_consensus.cc:2258] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:41:05.728813 30940 raft_consensus.cc:2229] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:41:05.729274 30942 raft_consensus.cc:2258] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:41:05.729481 30940 raft_consensus.cc:2258] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:41:05.731307 30941 ts_tablet_manager.cc:1905] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:41:05.731297 30942 ts_tablet_manager.cc:1905] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:41:05.731632 30940 ts_tablet_manager.cc:1905] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:41:05.738345 30942 ts_tablet_manager.cc:1918] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:41:05.738562 30942 log.cc:1196] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/786a68410b774de9a8c2191ecfa96f90
I20260808 23:41:05.738686 30941 ts_tablet_manager.cc:1918] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:41:05.738852 30941 log.cc:1196] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/786a68410b774de9a8c2191ecfa96f90
I20260808 23:41:05.739308 30940 ts_tablet_manager.cc:1918] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:41:05.739403 30941 ts_tablet_manager.cc:1939] T 786a68410b774de9a8c2191ecfa96f90 P d9dd5b7ed2204f90bebca1481a748a39: Deleting consensus metadata
I20260808 23:41:05.739485 30940 log.cc:1196] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/786a68410b774de9a8c2191ecfa96f90
I20260808 23:41:05.739462 30942 ts_tablet_manager.cc:1939] T 786a68410b774de9a8c2191ecfa96f90 P fa629fcb60df4b509bcd9ced8a309bbe: Deleting consensus metadata
I20260808 23:41:05.740078 30940 ts_tablet_manager.cc:1939] T 786a68410b774de9a8c2191ecfa96f90 P fcbbb0005e0d4fb2939154c289c775fe: Deleting consensus metadata
I20260808 23:41:05.741295 30368 catalog_manager.cc:4841] TS d9dd5b7ed2204f90bebca1481a748a39 (127.18.186.130:44355): tablet 786a68410b774de9a8c2191ecfa96f90 (table test-table [id=1a0592e806c046f88baffd6b751c921a]) successfully deleted
I20260808 23:41:05.741686 30368 catalog_manager.cc:4841] TS fa629fcb60df4b509bcd9ced8a309bbe (127.18.186.129:39031): tablet 786a68410b774de9a8c2191ecfa96f90 (table test-table [id=1a0592e806c046f88baffd6b751c921a]) successfully deleted
I20260808 23:41:05.741693 30370 catalog_manager.cc:4841] TS fcbbb0005e0d4fb2939154c289c775fe (127.18.186.131:34879): tablet 786a68410b774de9a8c2191ecfa96f90 (table test-table [id=1a0592e806c046f88baffd6b751c921a]) successfully deleted
Aug 08 23:41:05 dist-test-slave-5k54 krb5kdc[30333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:41:05.754765 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.718896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.18.186.129:39031)
0808 23:41:05.719151 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.719160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.728300 (+ 9140us) server_negotiation.cc:407] Connection header received
0808 23:41:05.735017 (+ 6717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.735024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.735109 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.735199 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:05.737061 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.738149 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.741806 (+ 3657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.742444 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.742544 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.747556 (+ 5012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:05.747572 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:05.747590 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:05.747634 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:05.751335 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:05.752229 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:05.752238 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:05.752242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:05.752420 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:05.752846 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:05.752853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:05.752856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:05.753339 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:05.753650 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.753933 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.754417 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:41:05.755558 30588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:05.868594 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 30451
I20260808 23:41:05.888111 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 30602
Aug 08 23:41:05 dist-test-slave-5k54 krb5kdc[30333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:41:05.905171 30908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:05.888995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40526 (local address 127.18.186.130:44355)
0808 23:41:05.889223 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:05.889230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:05.889252 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:41:05.889348 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:05.889354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:05.889448 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:05.889560 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:05.890736 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.891929 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.893154 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:05.893765 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:05.893904 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:05.898236 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:05.898252 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:05.898272 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:05.898314 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:05.902242 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:05.902860 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:05.902869 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:05.902880 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:05.903034 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:05.903474 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:05.903479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:05.903482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:05.903986 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:05.904310 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:05.904551 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 23:41:05.904791 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 23:41:05.905877 30736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:06.060295 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 30602
I20260808 23:41:06.077797 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 30748
Aug 08 23:41:06 dist-test-slave-5k54 krb5kdc[30333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:41:06.094229 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:06.078746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.18.186.131:34879)
0808 23:41:06.078964 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:06.078970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:06.078991 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:41:06.079142 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:06.079155 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:06.079222 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:06.079333 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:06.080626 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:06.081387 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:06.082662 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:06.083104 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:06.083253 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:06.088011 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:06.088024 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:06.088045 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:06.088096 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:06.091332 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:06.092064 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:06.092072 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:06.092075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:06.092204 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:06.092577 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:06.092584 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:06.092592 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:06.092988 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:06.093392 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:06.093545 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 23:41:06.093768 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:41:06.095077 30884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:06.218780 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 30748
I20260808 23:41:06.237560 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 30349
I20260808 23:41:06.238658 30417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:06.349690 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 30349
2026-08-08T23:41:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4468 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:41:06 dist-test-slave-5k54 krb5kdc[30951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:41:06 dist-test-slave-5k54 krb5kdc[30951](info): set up 2 sockets
Aug 08 23:41:06 dist-test-slave-5k54 krb5kdc[30951](info): commencing operation
krb5kdc: starting...
W20260808 23:41:07.526675 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:41:07 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:41:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:41:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.186.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.186.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/rangeradmin_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/rangeradmin_127.18.186.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.186.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.186.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/rangerlookup_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/rangerlookup_127.18.186.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.186.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.186.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/HTTP_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/HTTP_127.18.186.148@KRBTEST.COM.keytab.
I20260808 23:41:07.903259 19178 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskX1Oigb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/postgres -l logfile start
2026-08-08 23:41:10.450 UTC [30983] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 23:41:10.450 UTC [30983] LOG: listening on IPv4 address "127.18.186.148", port 39241
2026-08-08 23:41:10.454 UTC [30983] LOG: listening on Unix socket "/tmp/.s.PGSQL.39241"
2026-08-08 23:41:10.477 UTC [30986] LOG: database system was shut down at 2026-08-08 23:41:09 UTC
2026-08-08 23:41:10.481 UTC [30983] LOG: database system is ready to accept connections
I20260808 23:41:11.511484 19178 mini_postgres.cc:96] Postgres bound to 39241
2026-08-08 23:41:11.535 UTC [30995] FATAL: database "slave" does not exist
127.18.186.148:39241 - accepting connections
I20260808 23:41:11.536126 19178 mini_ranger.cc:161] Starting Ranger...
I20260808 23:41:11.566442 19178 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 23:41:12.067610 19178 mini_ranger.cc:87] Created ranger Postgres database
I20260808 23:41:12.067762 19178 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-admin
2026-08-08 23:41:12,399 [I] DB FLAVOR :POSTGRES
2026-08-08 23:41:12,400 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:41:12,400 [I] Checking connection..
2026-08-08 23:41:12,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 23:41:12,658 [I] Checking connection passed.
2026-08-08 23:41:12,658 [I] --------- Verifying version history table ---------
2026-08-08 23:41:12,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:41:12,926 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:41:12,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:41:13,182 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:41:13,182 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 23:41:13,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 23:41:13.437 UTC [31067] WARNING: there is no transaction in progress
2026-08-08 23:41:13,444 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 23:41:13,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:41:13,693 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 23:41:13,693 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:41:13,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 23:41:13,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 23:41:14,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5k54.c.gcp-upstream.internal','N') ;"
2026-08-08 23:41:14,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:41:14,693 [I] Table x_portal_user does not exist in database ranger
2026-08-08 23:41:14,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:41:14,942 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 23:41:14,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:41:15,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:41:15,488 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 23:41:15,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 23:41:16.411 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.427 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.443 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.459 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.477 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.493 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.556 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.563 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.576 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.586 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.596 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.609 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.618 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.625 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.631 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:16.637 UTC [31189] WARNING: there is no transaction in progress
2026-08-08 23:41:17,251 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 23:41:17,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 23:41:17,499 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 23:41:17,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:41:17,763 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 23:41:17,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:41:17,996 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 23:41:17,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:41:18,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:41:18,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5k54.c.gcp-upstream.internal';"
2026-08-08 23:41:18,767 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 23:41:18,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 23:41:19,028 [I] DB_PATCHES have already been applied
I20260808 23:41:19.034711 19178 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-admin:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskX1Oigb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 23:41:19.034807 19178 mini_ranger.cc:193] Using host: 127.18.186.148
I20260808 23:41:19.055039 19178 mini_ranger.cc:227] Ranger admin URL: http://127.18.186.148:58273
Aug 08, 2026 11:41:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:41:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:41:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:41:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:41:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:41:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:41:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 23:41:20.227298 19178 mini_ranger.cc:160] Time spent starting Ranger: real 8.691s user 0.006s sys 0.053s
Aug 08, 2026 11:41:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:41:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 11:41:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:41:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:41:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 23:41:41.922320 19178 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.186.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.186.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/rangerkms_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/rangerkms_127.18.186.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.186.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.186.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.186.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/HTTP_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.186.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/HTTP_127.18.186.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 23:41:42 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232502, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 23:41:42.144788 19178 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 23:41:42.173198 19178 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 23:41:43.085011 19178 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 23:41:43.085186 19178 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms
I20260808 23:41:43.085322 19178 mini_ranger_kms.cc:227] Using postgres at 127.18.186.148:39241
2026-08-08 23:41:43,564 [I] DB FLAVOR :POSTGRES
2026-08-08 23:41:43,564 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:41:43,564 [I] Checking connection
2026-08-08 23:41:43,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 23:41:43,803 [I] connection success
2026-08-08 23:41:43,804 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 23:41:43,804 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 23:41:43,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 23:41:44,046 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 23:41:44,046 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:41:44,047 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 23:41:44,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:39241/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 23:41:44,330 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 23:41:44.829674 19178 mini_ranger_kms.cc:310] Created kms service
I20260808 23:41:45.025627 19178 mini_ranger_kms.cc:326] Created kudu user
I20260808 23:41:45.071095 19178 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 23:41:45 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232502, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:41:45.465991 19178 mini_ranger_kms.cc:384] Added ranger policy
I20260808 23:41:45.467301 19178 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms:/tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 23:41:45.467382 19178 mini_ranger_kms.cc:240] Using host: 127.18.186.148
I20260808 23:41:45.492714 19178 mini_ranger_kms.cc:276] Ranger KMS PID: 31394
I20260808 23:41:45.492933 19178 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.186.148:34701
Aug 08, 2026 11:41:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:41:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:41:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:41:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:41:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:41:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 23:41:47.896560 19178 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.752s user 0.141s sys 0.378s
I20260808 23:41:47.896966 19178 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 23:41:47.897210 19178 mini_ranger_kms.cc:401] 127.18.186.148:34701/kms/v1/keys
Aug 08 23:41:47 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786232507, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.186.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 23:41:48 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232507, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.186.148@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
Aug 08 23:41:48 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232507, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.186.148@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
Aug 08 23:41:51 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:41:51.161325 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44453
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:37783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:44453
--ranger_config_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:34701/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:41:51.627575 31449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:51.627935 31449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:51.628120 31449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:51.637151 31449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:41:51.637329 31449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:51.637435 31449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:51.637562 31449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:41:51.637661 31449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:41:51.649797 31449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.186.148:34701/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:44453
--ranger_config_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:51.651378 31449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:51.652908 31449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:51.669476 31454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:51.669776 31457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:51.669904 31455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:51.673108 31449 server_base.cc:1034] running on GCE node
I20260808 23:41:51.674017 31449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:51.675674 31449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:51.676985 31449 hybrid_clock.cc:648] HybridClock initialized: now 1786232511676844 us; error 42 us; skew 500 ppm
I20260808 23:41:51.677467 31449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:41:51 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232511, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:41:51.681783 31449 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:41:51.683920 31449 webserver.cc:460] Webserver started at http://127.18.186.190:37985/ using document root <none> and password file <none>
I20260808 23:41:51.684674 31449 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:51.684866 31449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:51.685225 31449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:41:51 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232511, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:41:51.749436 31449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "b09d48c1dd5d46b09dbe924b4761dde5"
format_stamp: "Formatted at 2026-08-08 23:41:51 on dist-test-slave-5k54"
server_key: "77eee7cf6489978d5e8cd4db7e0c69ce"
server_key_iv: "ac9f1720a9f69e4aa7b86b2ddabbd707"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:51.750765 31449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "b09d48c1dd5d46b09dbe924b4761dde5"
format_stamp: "Formatted at 2026-08-08 23:41:51 on dist-test-slave-5k54"
server_key: "77eee7cf6489978d5e8cd4db7e0c69ce"
server_key_iv: "ac9f1720a9f69e4aa7b86b2ddabbd707"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:51.759703 31449 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.012s sys 0.000s
I20260808 23:41:51.782984 31464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:51.785167 31449 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260808 23:41:51.785390 31449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "b09d48c1dd5d46b09dbe924b4761dde5"
format_stamp: "Formatted at 2026-08-08 23:41:51 on dist-test-slave-5k54"
server_key: "77eee7cf6489978d5e8cd4db7e0c69ce"
server_key_iv: "ac9f1720a9f69e4aa7b86b2ddabbd707"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:51.785750 31449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:51.808382 31449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:51.815786 31449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:51.816337 31449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:51.859182 31449 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:44453
I20260808 23:41:51.859256 31523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:44453 every 8 connection(s)
I20260808 23:41:51.861446 31449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:41:51.863526 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 31449
I20260808 23:41:51.863850 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:41:51.864900 19178 external_mini_cluster.cc:1428] Setting key 5dc4cde54ea3bda774a6fef1542643e4
I20260808 23:41:51.869405 31524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:51.888461 31524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5: Bootstrap starting.
Aug 08 23:41:51 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:41:51.893414 31524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:51.896081 31524 log.cc:824] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:51.900705 31524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5: No bootstrap required, opened a new log
I20260808 23:41:51.904107 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:51.872958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.18.186.190:44453)
0808 23:41:51.874363 (+ 1405us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:51.874378 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:51.874458 (+ 80us) server_negotiation.cc:407] Connection header received
0808 23:41:51.878040 (+ 3582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:51.878070 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:51.878649 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:51.880918 (+ 2269us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:51.881502 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:51.884168 (+ 2666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:51.886073 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:51.887326 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:51.888836 (+ 1510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:51.892751 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:51.892796 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:51.892817 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:51.892865 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:51.898523 (+ 5658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:51.899452 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:51.899466 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:51.899474 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:51.899685 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:51.900220 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:51.900224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:51.900227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:51.900997 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:51.901393 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:51.901877 (+ 484us) server_negotiation.cc:299] Negotiation successful
0808 23:41:51.902292 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1074,"thread_start_us":731,"threads_started":1}
I20260808 23:41:51.913852 31524 raft_consensus.cc:348] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09d48c1dd5d46b09dbe924b4761dde5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44453 } }
I20260808 23:41:51.914309 31524 raft_consensus.cc:374] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:51.914505 31524 raft_consensus.cc:729] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09d48c1dd5d46b09dbe924b4761dde5, State: Initialized, Role: FOLLOWER
I20260808 23:41:51.915174 31524 consensus_queue.cc:260] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09d48c1dd5d46b09dbe924b4761dde5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44453 } }
I20260808 23:41:51.915457 31524 raft_consensus.cc:388] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:41:51.915647 31524 raft_consensus.cc:482] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:41:51.915832 31524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:51.919091 31524 raft_consensus.cc:504] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09d48c1dd5d46b09dbe924b4761dde5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44453 } }
I20260808 23:41:51.919822 31524 leader_election.cc:302] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b09d48c1dd5d46b09dbe924b4761dde5; no voters:
I20260808 23:41:51.921190 31524 leader_election.cc:288] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:41:51.921446 31530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:41:51.923202 31530 raft_consensus.cc:686] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [term 1 LEADER]: Becoming Leader. State: Replica: b09d48c1dd5d46b09dbe924b4761dde5, State: Running, Role: LEADER
I20260808 23:41:51.923910 31530 consensus_queue.cc:237] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09d48c1dd5d46b09dbe924b4761dde5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44453 } }
I20260808 23:41:51.924342 31524 sys_catalog.cc:564] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:41:51.934332 31531 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b09d48c1dd5d46b09dbe924b4761dde5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09d48c1dd5d46b09dbe924b4761dde5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44453 } } }
I20260808 23:41:51.935482 31531 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:41:51.934670 31532 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b09d48c1dd5d46b09dbe924b4761dde5. Latest consensus state: current_term: 1 leader_uuid: "b09d48c1dd5d46b09dbe924b4761dde5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09d48c1dd5d46b09dbe924b4761dde5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44453 } } }
I20260808 23:41:51.936702 31532 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:41:51.949213 31538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:41:51.958843 31538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:41:51.967625 31524 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 23:41:52.007836 31538 catalog_manager.cc:1347] Generated new cluster ID: 175854f802134cb7ac2d5af5cc3b32c0
I20260808 23:41:52.008018 31538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:41:52.029731 31538 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:41:52.031416 31538 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:41:52.042074 31538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5: Generated new TSK 0
I20260808 23:41:52.042958 31538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:41:52.953353 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:34701/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44453
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:37783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:41:53 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786232513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 23:41:53.687933 31573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:53.688290 31573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:53.688524 31573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:53.703052 31573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:53.703399 31573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:53.703794 31573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:41:53.722034 31573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.186.148:34701/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:53.724567 31573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:53.726660 31573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:53.755188 31585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:53.761433 31584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:53.761866 31587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:53.763526 31573 server_base.cc:1034] running on GCE node
I20260808 23:41:53.764652 31573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:53.766287 31573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:53.767562 31573 hybrid_clock.cc:648] HybridClock initialized: now 1786232513767495 us; error 50 us; skew 500 ppm
I20260808 23:41:53.768007 31573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:41:53 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:41:53.772964 31573 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:41:53.780614 31573 webserver.cc:460] Webserver started at http://127.18.186.129:40217/ using document root <none> and password file <none>
I20260808 23:41:53.782264 31573 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:53.782572 31573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:53.783226 31573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:41:53 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:41:53.810470 31573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "1208828499a5435bacb53f254c2eaf88"
format_stamp: "Formatted at 2026-08-08 23:41:53 on dist-test-slave-5k54"
server_key: "0f2e510742e4431bbdb970437c56d265"
server_key_iv: "a78d46074ce8bb96f73ba2dc65273e86"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:53.812101 31573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "1208828499a5435bacb53f254c2eaf88"
format_stamp: "Formatted at 2026-08-08 23:41:53 on dist-test-slave-5k54"
server_key: "0f2e510742e4431bbdb970437c56d265"
server_key_iv: "a78d46074ce8bb96f73ba2dc65273e86"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:53.828123 31573 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.002s sys 0.012s
I20260808 23:41:53.866199 31594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:53.868585 31573 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.005s sys 0.012s
I20260808 23:41:53.869138 31573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "1208828499a5435bacb53f254c2eaf88"
format_stamp: "Formatted at 2026-08-08 23:41:53 on dist-test-slave-5k54"
server_key: "0f2e510742e4431bbdb970437c56d265"
server_key_iv: "a78d46074ce8bb96f73ba2dc65273e86"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:53.869676 31573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:53.934176 31573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:53.941797 31573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:53.942354 31573 kserver.cc:163] Server-wide thread pool size limit: 3276
Aug 08 23:41:53 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:41:53.951028 31573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:53.959937 31573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:53.960145 31573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 23:41:53.960486 31573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:53.960673 31573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:54.286978 31573 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:41169
I20260808 23:41:54.287061 31717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:41169 every 8 connection(s)
I20260808 23:41:54.289073 31573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:41:54.296008 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 31573
I20260808 23:41:54.296283 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:41:54.297416 19178 external_mini_cluster.cc:1428] Setting key 25047b2d68ce693197935a69567cf84f
Aug 08 23:41:54 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:41:54.349382 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:54.295646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:50189 (local address 127.18.186.190:44453)
0808 23:41:54.319765 (+ 24119us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:54.319787 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:54.325860 (+ 6073us) server_negotiation.cc:407] Connection header received
0808 23:41:54.325971 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:54.325978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:54.326059 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:54.326388 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:54.329416 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:54.330807 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:54.333677 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:54.334498 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:54.334596 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:54.340926 (+ 6330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:54.340933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:54.340938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:54.341022 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:54.345518 (+ 4496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:54.346275 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:54.346282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:54.346290 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:54.346437 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:54.346896 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:54.346900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:54.346904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:54.347374 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:54.347626 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:54.348755 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0808 23:41:54.348976 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23903,"thread_start_us":23833,"threads_started":1}
I20260808 23:41:54.351258 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:54.294432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44453 (local address 127.18.186.129:50189)
0808 23:41:54.321141 (+ 26709us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:54.321209 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:54.324397 (+ 3188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:54.326855 (+ 2458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:54.326868 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:54.327840 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:54.329202 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:54.329229 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:54.331020 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:54.331026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:54.333507 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:54.333523 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:54.335158 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:54.335167 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:54.335668 (+ 501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:54.336749 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:41:54.336781 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:54.340621 (+ 3840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:54.345676 (+ 5055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:54.345682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:54.345696 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:54.346125 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:54.346555 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:54.346560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:54.346563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:54.346762 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:54.347640 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:54.347649 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:54.348191 (+ 542us) client_negotiation.cc:769] Sending connection context
0808 23:41:54.348639 (+ 448us) client_negotiation.cc:240] Negotiation successful
0808 23:41:54.349294 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26322,"thread_start_us":30259,"threads_started":1}
I20260808 23:41:54.353569 31719 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44453
I20260808 23:41:54.354024 31719 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:54.355132 31719 heartbeater.cc:511] Master 127.18.186.190:44453 requested a full tablet report, sending...
I20260808 23:41:54.358618 31483 ts_manager.cc:194] Registered new tserver with Master: 1208828499a5435bacb53f254c2eaf88 (127.18.186.129:41169)
I20260808 23:41:54.361045 31483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:50189
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
W20260808 23:41:54.423043 19178 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.186.130: real 0.100s user 0.001s sys 0.045s
I20260808 23:41:54.424096 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:34701/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44453
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:37783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:41:54 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
W20260808 23:41:55.074581 31727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:55.074981 31727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:55.075251 31727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:55.085903 31727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:55.086138 31727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:55.086354 31727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:41:55.099786 31727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.186.148:34701/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:55.101454 31727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:55.102981 31727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:55.119484 31742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:55.120533 31745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:55.120561 31743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:55.122311 31727 server_base.cc:1034] running on GCE node
I20260808 23:41:55.123008 31727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:55.124228 31727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:55.125455 31727 hybrid_clock.cc:648] HybridClock initialized: now 1786232515125397 us; error 43 us; skew 500 ppm
I20260808 23:41:55.125852 31727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:41:55 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232515, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:41:55.130203 31727 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:41:55.132268 31727 webserver.cc:460] Webserver started at http://127.18.186.130:43799/ using document root <none> and password file <none>
I20260808 23:41:55.133117 31727 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:55.133313 31727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:55.133723 31727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:41:55 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232515, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:41:55.151496 31727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "c4af284c5f5b41c5941038cf6ae4d400"
format_stamp: "Formatted at 2026-08-08 23:41:55 on dist-test-slave-5k54"
server_key: "24435823295098d30d7a048dd6f3a612"
server_key_iv: "b8d60a1356537663ab85cde881c4aafa"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:55.152336 31727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "c4af284c5f5b41c5941038cf6ae4d400"
format_stamp: "Formatted at 2026-08-08 23:41:55 on dist-test-slave-5k54"
server_key: "24435823295098d30d7a048dd6f3a612"
server_key_iv: "b8d60a1356537663ab85cde881c4aafa"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:55.158787 31727 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260808 23:41:55.177616 31752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:55.179507 31727 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260808 23:41:55.179764 31727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "c4af284c5f5b41c5941038cf6ae4d400"
format_stamp: "Formatted at 2026-08-08 23:41:55 on dist-test-slave-5k54"
server_key: "24435823295098d30d7a048dd6f3a612"
server_key_iv: "b8d60a1356537663ab85cde881c4aafa"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:55.180086 31727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:55.226591 31727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:55.231912 31727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:55.232545 31727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:55.234230 31727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:55.236290 31727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:55.236389 31727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:55.236656 31727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:55.236748 31727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:55.339563 31727 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:38601
I20260808 23:41:55.339681 31881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:38601 every 8 connection(s)
I20260808 23:41:55.340993 31727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:41:55.344627 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 31727
I20260808 23:41:55.344823 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:41:55.345562 19178 external_mini_cluster.cc:1428] Setting key 0e697209037ab2f927502ea7fcd98c38
I20260808 23:41:55.365375 31719 heartbeater.cc:503] Master 127.18.186.190:44453 was elected leader, sending a full tablet report...
Aug 08 23:41:55 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232515, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:41:55.379863 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:55.344705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:47975 (local address 127.18.186.190:44453)
0808 23:41:55.350143 (+ 5438us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:55.350156 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:55.351632 (+ 1476us) server_negotiation.cc:407] Connection header received
0808 23:41:55.355626 (+ 3994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:55.355632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:55.355713 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:55.355918 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:55.358981 (+ 3063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:55.360778 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:55.363996 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:55.364706 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:55.364850 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:55.370519 (+ 5669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:55.370526 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:55.370531 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:55.370604 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:55.375198 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:55.376376 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:55.376382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:55.376386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:55.376624 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:55.377304 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:55.377310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:55.377313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:55.377786 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:55.377953 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:55.379040 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0808 23:41:55.379273 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5134,"thread_start_us":4955,"threads_started":1}
I20260808 23:41:55.381232 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:55.345669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44453 (local address 127.18.186.130:47975)
0808 23:41:55.351397 (+ 5728us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:55.351473 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:55.355215 (+ 3742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:55.356291 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:55.356305 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:55.357353 (+ 1048us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:55.358740 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:55.358773 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:55.360943 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:55.360950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:55.363831 (+ 2881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:55.363850 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:55.365060 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:55.365070 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:55.365488 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:55.366519 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:41:55.366599 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:55.370212 (+ 3613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:55.375412 (+ 5200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:55.375424 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:55.375468 (+ 44us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:55.376224 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:55.376817 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:55.376822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:55.376825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:55.377151 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:55.377915 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:55.377927 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:55.378486 (+ 559us) client_negotiation.cc:769] Sending connection context
0808 23:41:55.378886 (+ 400us) client_negotiation.cc:240] Negotiation successful
0808 23:41:55.379540 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5404,"thread_start_us":5065,"threads_started":1}
I20260808 23:41:55.383602 31882 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44453
I20260808 23:41:55.383984 31882 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:55.385352 31882 heartbeater.cc:511] Master 127.18.186.190:44453 requested a full tablet report, sending...
I20260808 23:41:55.387389 31483 ts_manager.cc:194] Registered new tserver with Master: c4af284c5f5b41c5941038cf6ae4d400 (127.18.186.130:38601)
I20260808 23:41:55.388633 31483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:47975
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:41:55.428354 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:34701/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44453
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:37783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:41:56.000403 31889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:41:56.000741 31889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:41:56.000895 31889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:41:56.010509 31889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:41:56.010768 31889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:41:56.010975 31889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:41:56.024724 31889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.186.148:34701/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:41:56.026260 31889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:41:56.027683 31889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:41:56.045073 31894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:56.046303 31897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:41:56.046525 31895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:41:56.048580 31889 server_base.cc:1034] running on GCE node
I20260808 23:41:56.049268 31889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:41:56.050453 31889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:41:56.051707 31889 hybrid_clock.cc:648] HybridClock initialized: now 1786232516051616 us; error 55 us; skew 500 ppm
I20260808 23:41:56.052130 31889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:41:56 dist-test-slave-5k54 krb5kdc[30951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232516, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:41:56.056838 31889 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:41:56.058856 31889 webserver.cc:460] Webserver started at http://127.18.186.131:45335/ using document root <none> and password file <none>
I20260808 23:41:56.059890 31889 fs_manager.cc:359] Metadata directory not provided
I20260808 23:41:56.060082 31889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:41:56.060510 31889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:41:56 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232516, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:41:56.077853 31889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "59213ccea1124a8ead9530c94427bfb7"
format_stamp: "Formatted at 2026-08-08 23:41:56 on dist-test-slave-5k54"
server_key: "2d2e882619e13d37d6b32fd478486035"
server_key_iv: "feff307fc8a4d2cf3f15c3fffd315ebb"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:56.078953 31889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "59213ccea1124a8ead9530c94427bfb7"
format_stamp: "Formatted at 2026-08-08 23:41:56 on dist-test-slave-5k54"
server_key: "2d2e882619e13d37d6b32fd478486035"
server_key_iv: "feff307fc8a4d2cf3f15c3fffd315ebb"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:56.086495 31889 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.001s
I20260808 23:41:56.106825 31904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:56.109006 31889 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260808 23:41:56.109279 31889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "59213ccea1124a8ead9530c94427bfb7"
format_stamp: "Formatted at 2026-08-08 23:41:56 on dist-test-slave-5k54"
server_key: "2d2e882619e13d37d6b32fd478486035"
server_key_iv: "feff307fc8a4d2cf3f15c3fffd315ebb"
server_key_version: "kuduclusterkey@0"
I20260808 23:41:56.109594 31889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:41:56.136075 31889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:41:56.141738 31889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:41:56.142302 31889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:41:56.144189 31889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:41:56.146613 31889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:41:56.146798 31889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:56.147053 31889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:41:56.147217 31889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:41:56.241165 31889 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:36867
I20260808 23:41:56.241344 32027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:36867 every 8 connection(s)
I20260808 23:41:56.242893 31889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:41:56.249353 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 31889
I20260808 23:41:56.249547 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:41:56.250108 19178 external_mini_cluster.cc:1428] Setting key 0704a20c33cb171dfc9905fe52624a1f
Aug 08 23:41:56 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232516, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:41:56.271317 32030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.246887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:59793 (local address 127.18.186.190:44453)
0808 23:41:56.247716 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.247720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.249058 (+ 1338us) server_negotiation.cc:407] Connection header received
0808 23:41:56.251843 (+ 2785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.251849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.251943 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.252147 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:56.254290 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.255526 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.257703 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.258262 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.258407 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.263203 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:56.263209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:56.263212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:56.263245 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:56.267856 (+ 4611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:56.268664 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:56.268670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:56.268673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:56.268837 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:56.269391 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:56.269396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:56.269400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:56.269757 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:56.269967 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.270659 (+ 692us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.270896 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":484,"threads_started":1}
I20260808 23:41:56.272575 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.247805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44453 (local address 127.18.186.131:59793)
0808 23:41:56.248821 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:56.248896 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:56.251477 (+ 2581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:56.252498 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:56.252511 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:56.253330 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:41:56.254121 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.254140 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.255664 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.255670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.257515 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.257528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.258658 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.258667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.258986 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.259912 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:41:56.259935 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:41:56.262916 (+ 2981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:41:56.267990 (+ 5074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:56.267999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:56.268022 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:56.268534 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:56.268958 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:41:56.268963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:41:56.268967 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:41:56.269256 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:41:56.269914 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:41:56.269922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:41:56.270287 (+ 365us) client_negotiation.cc:769] Sending connection context
0808 23:41:56.270580 (+ 293us) client_negotiation.cc:240] Negotiation successful
0808 23:41:56.271209 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":511,"threads_started":1}
I20260808 23:41:56.274791 32028 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44453
I20260808 23:41:56.275182 32028 heartbeater.cc:464] Registering TS with master...
I20260808 23:41:56.276041 32028 heartbeater.cc:511] Master 127.18.186.190:44453 requested a full tablet report, sending...
I20260808 23:41:56.278131 31483 ts_manager.cc:194] Registered new tserver with Master: 59213ccea1124a8ead9530c94427bfb7 (127.18.186.131:36867)
I20260808 23:41:56.279534 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:41:56.279618 31483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:59793
I20260808 23:41:56.303649 32030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.286358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.18.186.190:44453)
0808 23:41:56.286600 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.286607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.287582 (+ 975us) server_negotiation.cc:407] Connection header received
0808 23:41:56.287817 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.287821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.287894 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.287975 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:56.290426 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.291230 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.293159 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.293723 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.293856 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.296945 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:56.296950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:56.296953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:56.296984 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:56.300792 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:56.301515 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:56.301518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:56.301521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:56.301716 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:56.302129 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:56.302133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:56.302135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:56.302483 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:56.302716 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.302989 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.303220 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:41:56.308439 31483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:41:56.319311 31483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:41:56.370460 32043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.336134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:36867 (local address 127.0.0.1:45692)
0808 23:41:56.349099 (+ 12965us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:56.349149 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:56.349442 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:56.359103 (+ 9661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:56.359111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:56.359142 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:56.360617 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.360633 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.362961 (+ 2328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.362969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.365571 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.365584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.368232 (+ 2648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.368250 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.368617 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.368659 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 23:41:56.368802 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 23:41:56.368977 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12567,"mutex_wait_us":44,"thread_start_us":13490,"threads_started":1}
I20260808 23:41:56.370679 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.352102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.18.186.129:41169)
0808 23:41:56.354365 (+ 2263us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.354378 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.356530 (+ 2152us) server_negotiation.cc:407] Connection header received
0808 23:41:56.356949 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.356960 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.357303 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.357582 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:56.359831 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.362097 (+ 2266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.366077 (+ 3980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.368054 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.369132 (+ 1078us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.369319 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.369383 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.369549 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2046,"thread_start_us":1838,"threads_started":1}
I20260808 23:41:56.371016 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.336240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:38601 (local address 127.0.0.1:48530)
0808 23:41:56.355037 (+ 18797us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:56.355063 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:56.355220 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:56.358451 (+ 3231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:56.358456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:56.358481 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:56.359474 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.359498 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.362468 (+ 2970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.362475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.366723 (+ 4248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.366735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.369407 (+ 2672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.369419 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.369653 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.369685 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:41:56.369765 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 23:41:56.369889 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18591,"mutex_wait_us":23,"thread_start_us":19089,"threads_started":1}
I20260808 23:41:56.371942 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.336226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48530 (local address 127.18.186.130:38601)
0808 23:41:56.357901 (+ 21675us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.357911 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.357948 (+ 37us) server_negotiation.cc:407] Connection header received
0808 23:41:56.358085 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.358091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.358241 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.358487 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:56.359752 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.362275 (+ 2523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.366866 (+ 4591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.368511 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.368882 (+ 371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.369045 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.371152 (+ 2107us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.371397 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21441,"thread_start_us":21196,"threads_started":1}
I20260808 23:41:56.372005 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.351251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:41169 (local address 127.0.0.1:59766)
0808 23:41:56.356415 (+ 5164us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:56.356431 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:56.356625 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:56.358135 (+ 1510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:56.358143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:56.358175 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:56.358966 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.358985 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.362247 (+ 3262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.362253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.365920 (+ 3667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.365934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.368331 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.368340 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.368586 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.368650 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 23:41:56.368771 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 23:41:56.368969 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5014,"mutex_wait_us":100,"thread_start_us":4838,"threads_started":1}
I20260808 23:41:56.372915 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.336989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.18.186.131:36867)
0808 23:41:56.358377 (+ 21388us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.358387 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.358450 (+ 63us) server_negotiation.cc:407] Connection header received
0808 23:41:56.358546 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.358554 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.358841 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.359117 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:56.360953 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.362802 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.365734 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.367571 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.367777 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.367991 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.371792 (+ 3801us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.372235 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21122,"thread_start_us":20758,"threads_started":1}
I20260808 23:41:56.376794 31958 tablet_service.cc:1467] Processing CreateTablet for tablet bf11733f824e43ccaaa9cd2455cd3327 (DEFAULT_TABLE table=test-table [id=f537bf94b7b34030b0f8ed8f14c8eacf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:41:56.376786 31647 tablet_service.cc:1467] Processing CreateTablet for tablet bf11733f824e43ccaaa9cd2455cd3327 (DEFAULT_TABLE table=test-table [id=f537bf94b7b34030b0f8ed8f14c8eacf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:41:56.377374 31807 tablet_service.cc:1467] Processing CreateTablet for tablet bf11733f824e43ccaaa9cd2455cd3327 (DEFAULT_TABLE table=test-table [id=f537bf94b7b34030b0f8ed8f14c8eacf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:41:56.378623 31958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf11733f824e43ccaaa9cd2455cd3327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:56.379027 31647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf11733f824e43ccaaa9cd2455cd3327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:56.379637 31807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf11733f824e43ccaaa9cd2455cd3327. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:41:56.391628 31882 heartbeater.cc:503] Master 127.18.186.190:44453 was elected leader, sending a full tablet report...
I20260808 23:41:56.398984 32049 tablet_bootstrap.cc:492] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Bootstrap starting.
I20260808 23:41:56.404165 32049 tablet_bootstrap.cc:654] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:56.405079 32051 tablet_bootstrap.cc:492] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Bootstrap starting.
I20260808 23:41:56.406569 32050 tablet_bootstrap.cc:492] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Bootstrap starting.
I20260808 23:41:56.406772 32049 log.cc:824] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:56.411340 32051 tablet_bootstrap.cc:654] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:56.412863 32051 log.cc:824] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:56.413697 32050 tablet_bootstrap.cc:654] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Neither blocks nor log segments found. Creating new log.
I20260808 23:41:56.415786 32050 log.cc:824] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Log is configured to *not* fsync() on all Append() calls
I20260808 23:41:56.417652 32049 tablet_bootstrap.cc:492] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: No bootstrap required, opened a new log
I20260808 23:41:56.417833 32051 tablet_bootstrap.cc:492] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: No bootstrap required, opened a new log
I20260808 23:41:56.418197 32051 ts_tablet_manager.cc:1397] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260808 23:41:56.418169 32049 ts_tablet_manager.cc:1397] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.012s
I20260808 23:41:56.419993 32050 tablet_bootstrap.cc:492] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: No bootstrap required, opened a new log
I20260808 23:41:56.420359 32050 ts_tablet_manager.cc:1397] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260808 23:41:56.430792 32050 raft_consensus.cc:348] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.431309 32050 raft_consensus.cc:374] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:56.431524 32050 raft_consensus.cc:729] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1208828499a5435bacb53f254c2eaf88, State: Initialized, Role: FOLLOWER
I20260808 23:41:56.432334 32050 consensus_queue.cc:260] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.433719 32051 raft_consensus.cc:348] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.433954 32049 raft_consensus.cc:348] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.434309 32051 raft_consensus.cc:374] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:56.434465 32049 raft_consensus.cc:374] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:41:56.434479 32051 raft_consensus.cc:729] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4af284c5f5b41c5941038cf6ae4d400, State: Initialized, Role: FOLLOWER
I20260808 23:41:56.434592 32049 raft_consensus.cc:729] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59213ccea1124a8ead9530c94427bfb7, State: Initialized, Role: FOLLOWER
I20260808 23:41:56.434648 32050 ts_tablet_manager.cc:1428] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260808 23:41:56.435240 32051 consensus_queue.cc:260] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.435396 32049 consensus_queue.cc:260] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.437323 32028 heartbeater.cc:503] Master 127.18.186.190:44453 was elected leader, sending a full tablet report...
I20260808 23:41:56.438212 32049 ts_tablet_manager.cc:1428] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Time spent starting tablet: real 0.020s user 0.014s sys 0.007s
I20260808 23:41:56.442492 32051 ts_tablet_manager.cc:1428] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Time spent starting tablet: real 0.024s user 0.008s sys 0.017s
W20260808 23:41:56.495855 32029 tablet.cc:2369] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:41:56.564782 31720 tablet.cc:2369] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:41:56.594452 31883 tablet.cc:2369] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:41:56.655143 32055 raft_consensus.cc:482] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:41:56.655515 32055 raft_consensus.cc:504] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.657730 32055 leader_election.cc:288] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4af284c5f5b41c5941038cf6ae4d400 (127.18.186.130:38601), 59213ccea1124a8ead9530c94427bfb7 (127.18.186.131:36867)
I20260808 23:41:56.668314 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.658298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:36053 (local address 127.18.186.131:36867)
0808 23:41:56.658476 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.658481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.660496 (+ 2015us) server_negotiation.cc:407] Connection header received
0808 23:41:56.660769 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.660775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.660856 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.660967 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:56.662115 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.663012 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.665731 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.667044 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.667286 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.667401 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.667813 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.667952 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 23:41:56.668684 32059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.658912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:36867 (local address 127.18.186.129:36053)
0808 23:41:56.660289 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:56.660306 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:56.660601 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:56.660995 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:56.661000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:56.661034 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:56.661944 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.661976 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.663166 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.663184 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.665570 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.665583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.667406 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.667415 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.667648 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.667673 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:41:56.667787 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 23:41:56.667933 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1221,"spinlock_wait_cycles":23296,"thread_start_us":1984,"threads_started":1}
I20260808 23:41:56.669314 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.658261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:58321 (local address 127.18.186.130:38601)
0808 23:41:56.658481 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.658485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.661655 (+ 3170us) server_negotiation.cc:407] Connection header received
0808 23:41:56.661873 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.661885 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.661964 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.662064 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:41:56.663043 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.664147 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.666757 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.668023 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.668227 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.668324 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.668969 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.669119 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:41:56.669625 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.658929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:38601 (local address 127.18.186.129:58321)
0808 23:41:56.661488 (+ 2559us) negotiation.cc:107] Waiting for socket to connect
0808 23:41:56.661506 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:41:56.661722 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:41:56.662082 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:41:56.662086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:41:56.662110 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:41:56.662849 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.662859 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.664302 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.664308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.666561 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:41:56.666574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.668455 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.668465 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:41:56.668766 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.668813 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 23:41:56.668922 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 23:41:56.669056 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2308,"mutex_wait_us":4,"thread_start_us":944,"threads_started":1}
I20260808 23:41:56.670099 31980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf11733f824e43ccaaa9cd2455cd3327" candidate_uuid: "1208828499a5435bacb53f254c2eaf88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59213ccea1124a8ead9530c94427bfb7" is_pre_election: true
I20260808 23:41:56.670841 31980 raft_consensus.cc:2454] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1208828499a5435bacb53f254c2eaf88 in term 0.
I20260808 23:41:56.671480 31828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf11733f824e43ccaaa9cd2455cd3327" candidate_uuid: "1208828499a5435bacb53f254c2eaf88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4af284c5f5b41c5941038cf6ae4d400" is_pre_election: true
I20260808 23:41:56.671926 31597 leader_election.cc:302] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1208828499a5435bacb53f254c2eaf88, 59213ccea1124a8ead9530c94427bfb7; no voters:
I20260808 23:41:56.672391 31828 raft_consensus.cc:2454] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1208828499a5435bacb53f254c2eaf88 in term 0.
I20260808 23:41:56.672550 32055 raft_consensus.cc:2781] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:41:56.672824 32055 raft_consensus.cc:482] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:41:56.673004 32055 raft_consensus.cc:3037] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:56.676115 32055 raft_consensus.cc:504] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.676976 32055 leader_election.cc:288] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [CANDIDATE]: Term 1 election: Requested vote from peers c4af284c5f5b41c5941038cf6ae4d400 (127.18.186.130:38601), 59213ccea1124a8ead9530c94427bfb7 (127.18.186.131:36867)
I20260808 23:41:56.677575 31828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf11733f824e43ccaaa9cd2455cd3327" candidate_uuid: "1208828499a5435bacb53f254c2eaf88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4af284c5f5b41c5941038cf6ae4d400"
I20260808 23:41:56.677615 31980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf11733f824e43ccaaa9cd2455cd3327" candidate_uuid: "1208828499a5435bacb53f254c2eaf88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59213ccea1124a8ead9530c94427bfb7"
I20260808 23:41:56.677814 31828 raft_consensus.cc:3037] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:56.677886 31980 raft_consensus.cc:3037] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:41:56.680981 31980 raft_consensus.cc:2454] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1208828499a5435bacb53f254c2eaf88 in term 1.
I20260808 23:41:56.681066 31828 raft_consensus.cc:2454] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1208828499a5435bacb53f254c2eaf88 in term 1.
I20260808 23:41:56.681463 31597 leader_election.cc:302] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1208828499a5435bacb53f254c2eaf88, 59213ccea1124a8ead9530c94427bfb7; no voters:
I20260808 23:41:56.681878 32055 raft_consensus.cc:2781] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:41:56.682715 32055 raft_consensus.cc:686] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 1 LEADER]: Becoming Leader. State: Replica: 1208828499a5435bacb53f254c2eaf88, State: Running, Role: LEADER
I20260808 23:41:56.683713 32055 consensus_queue.cc:237] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } }
I20260808 23:41:56.689277 31482 catalog_manager.cc:5495] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1208828499a5435bacb53f254c2eaf88 (127.18.186.129). New cstate: current_term: 1 leader_uuid: "1208828499a5435bacb53f254c2eaf88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1208828499a5435bacb53f254c2eaf88" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 41169 } health_report { overall_health: HEALTHY } } }
I20260808 23:41:56.741190 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.734121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.18.186.129:41169)
0808 23:41:56.734432 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.734439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.734691 (+ 252us) server_negotiation.cc:407] Connection header received
0808 23:41:56.734960 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.734965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.735087 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.735262 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:41:56.735798 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.736754 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.738539 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.739206 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.739349 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.740033 (+ 684us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:41:56.740561 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:41:56.740735 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.740793 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.740875 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 23:41:56.752050 31980 raft_consensus.cc:1261] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 1 FOLLOWER]: Refusing update from remote peer 1208828499a5435bacb53f254c2eaf88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:41:56.752091 31828 raft_consensus.cc:1261] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 1 FOLLOWER]: Refusing update from remote peer 1208828499a5435bacb53f254c2eaf88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:41:56.753216 32060 consensus_queue.cc:1035] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4af284c5f5b41c5941038cf6ae4d400" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 38601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:41:56.753718 32055 consensus_queue.cc:1035] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59213ccea1124a8ead9530c94427bfb7" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:41:56.768242 32063 mvcc.cc:205] Tried to move back new op lower bound from 7316408388604022784 to 7316408388341153792. Current Snapshot: MvccSnapshot[applied={T|T < 7316408388604022784}]
I20260808 23:41:56.786911 31483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:41:56.787503 31483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:41:56.792671 31483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b09d48c1dd5d46b09dbe924b4761dde5: Sending DeleteTablet for 3 replicas of tablet bf11733f824e43ccaaa9cd2455cd3327
I20260808 23:41:56.794241 31958 tablet_service.cc:1514] Processing DeleteTablet for tablet bf11733f824e43ccaaa9cd2455cd3327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:41:56 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:45692
I20260808 23:41:56.794354 31807 tablet_service.cc:1514] Processing DeleteTablet for tablet bf11733f824e43ccaaa9cd2455cd3327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:41:56 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:48530
I20260808 23:41:56.794622 31647 tablet_service.cc:1514] Processing DeleteTablet for tablet bf11733f824e43ccaaa9cd2455cd3327 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:41:56 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:59766
I20260808 23:41:56.797240 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 31573
I20260808 23:41:56.810828 32077 tablet_replica.cc:330] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: stopping tablet replica
I20260808 23:41:56.811530 32077 raft_consensus.cc:2229] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:41:56.812876 32077 raft_consensus.cc:2258] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:41:56.815479 32075 tablet_replica.cc:330] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: stopping tablet replica
I20260808 23:41:56.815960 32075 raft_consensus.cc:2229] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:41:56.816440 32075 raft_consensus.cc:2258] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:41:56.817284 32074 tablet_replica.cc:330] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: stopping tablet replica
I20260808 23:41:56.817785 32074 raft_consensus.cc:2229] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:41:56.818225 32077 ts_tablet_manager.cc:1905] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:41:56.818267 32074 raft_consensus.cc:2258] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:41:56.819293 32075 ts_tablet_manager.cc:1905] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:41:56.821679 32074 ts_tablet_manager.cc:1905] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:41:56.827217 32075 ts_tablet_manager.cc:1918] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:41:56.827525 32075 log.cc:1196] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/bf11733f824e43ccaaa9cd2455cd3327
I20260808 23:41:56.828281 32075 ts_tablet_manager.cc:1939] T bf11733f824e43ccaaa9cd2455cd3327 P c4af284c5f5b41c5941038cf6ae4d400: Deleting consensus metadata
I20260808 23:41:56.830708 31465 catalog_manager.cc:4841] TS c4af284c5f5b41c5941038cf6ae4d400 (127.18.186.130:38601): tablet bf11733f824e43ccaaa9cd2455cd3327 (table test-table [id=f537bf94b7b34030b0f8ed8f14c8eacf]) successfully deleted
I20260808 23:41:56.830807 32074 ts_tablet_manager.cc:1918] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:41:56.831055 32074 log.cc:1196] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/bf11733f824e43ccaaa9cd2455cd3327
I20260808 23:41:56.831847 32074 ts_tablet_manager.cc:1939] T bf11733f824e43ccaaa9cd2455cd3327 P 59213ccea1124a8ead9530c94427bfb7: Deleting consensus metadata
I20260808 23:41:56.833901 31467 catalog_manager.cc:4841] TS 59213ccea1124a8ead9530c94427bfb7 (127.18.186.131:36867): tablet bf11733f824e43ccaaa9cd2455cd3327 (table test-table [id=f537bf94b7b34030b0f8ed8f14c8eacf]) successfully deleted
I20260808 23:41:56.834983 32077 ts_tablet_manager.cc:1918] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:41:56.835187 32077 log.cc:1196] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/bf11733f824e43ccaaa9cd2455cd3327
I20260808 23:41:56.835834 32077 ts_tablet_manager.cc:1939] T bf11733f824e43ccaaa9cd2455cd3327 P 1208828499a5435bacb53f254c2eaf88: Deleting consensus metadata
Aug 08 23:41:56 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:41:56.837806 31467 catalog_manager.cc:4841] TS 1208828499a5435bacb53f254c2eaf88 (127.18.186.129:41169): tablet bf11733f824e43ccaaa9cd2455cd3327 (table test-table [id=f537bf94b7b34030b0f8ed8f14c8eacf]) successfully deleted
I20260808 23:41:56.846628 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.798326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.18.186.129:41169)
0808 23:41:56.798518 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.798524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.814396 (+ 15872us) server_negotiation.cc:407] Connection header received
0808 23:41:56.814698 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.814704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.814800 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.814921 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:56.823594 (+ 8673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.825191 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.828119 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.829193 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.829377 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.838588 (+ 9211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:56.838599 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:56.838613 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:56.838673 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:56.842857 (+ 4184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:56.844070 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:56.844078 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:56.844080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:56.844353 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:56.844864 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:56.844868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:56.844870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:56.845241 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:56.845627 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.845990 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.846200 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 23:41:56.847834 31713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:56.958070 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 31573
I20260808 23:41:56.979377 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 31727
Aug 08 23:41:56 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:41:56.995723 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:56.980218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.18.186.130:38601)
0808 23:41:56.980387 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:56.980392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:56.980410 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:41:56.980557 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:56.980560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:56.980630 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:56.980716 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:56.981975 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.982659 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.984281 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:56.984726 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:56.984832 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:56.989187 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:56.989201 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:56.989216 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:56.989249 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:56.992033 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:56.992803 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:56.992809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:56.992812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:56.993025 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:56.993477 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:56.993481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:56.993484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:56.994198 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:56.994565 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:56.994958 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 23:41:56.995270 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:41:56.996490 31876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:57.103169 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 31727
I20260808 23:41:57.122262 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 31889
Aug 08 23:41:57 dist-test-slave-5k54 krb5kdc[30951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:41:57.137315 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:41:57.123245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.18.186.131:36867)
0808 23:41:57.123455 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:41:57.123462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:41:57.123481 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:41:57.123559 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:41:57.123563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:41:57.123629 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:41:57.123712 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:41:57.125120 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:57.125982 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:57.127236 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:41:57.127791 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:41:57.127886 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:41:57.132174 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:41:57.132189 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:41:57.132203 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:41:57.132240 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:41:57.134800 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:57.135499 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:57.135505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:57.135508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:57.135640 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:41:57.135958 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:41:57.135961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:41:57.135963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:41:57.136317 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:41:57.136519 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 23:41:57.136761 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 23:41:57.136932 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:41:57.137951 32022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:57.226980 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 31889
I20260808 23:41:57.245581 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 31449
I20260808 23:41:57.246806 31518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:41:57.361380 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 31449
I20260808 23:41:57.389359 19178 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 23:41:57.724268 19178 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 23:41:57.724553 19178 mini_ranger.cc:66] Stopping Ranger...
I20260808 23:41:58.068769 19178 mini_ranger.cc:68] Stopped Ranger
2026-08-08 23:41:58.069 UTC [30983] LOG: received smart shutdown request
2026-08-08 23:41:58.073 UTC [30983] LOG: background worker "logical replication launcher" (PID 30992) exited with exit code 1
2026-08-08 23:41:58.073 UTC [30987] LOG: shutting down
2026-08-08 23:41:58.233 UTC [30983] LOG: database system is shut down
2026-08-08T23:41:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52035 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:41:58 dist-test-slave-5k54 krb5kdc[32093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:41:58 dist-test-slave-5k54 krb5kdc[32093](info): set up 2 sockets
Aug 08 23:41:58 dist-test-slave-5k54 krb5kdc[32093](info): commencing operation
krb5kdc: starting...
W20260808 23:41:59.571640 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:41:59 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:41:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:41:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.186.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.186.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/rangeradmin_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/rangeradmin_127.18.186.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.186.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.186.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/rangerlookup_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/rangerlookup_127.18.186.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.186.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.186.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/HTTP_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/HTTP_127.18.186.148@KRBTEST.COM.keytab.
I20260808 23:41:59.946360 19178 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskX1Oigb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/postgres -l logfile start
2026-08-08 23:42:02.419 UTC [32135] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 23:42:02.420 UTC [32135] LOG: listening on IPv4 address "127.18.186.148", port 37209
2026-08-08 23:42:02.423 UTC [32135] LOG: listening on Unix socket "/tmp/.s.PGSQL.37209"
2026-08-08 23:42:02.447 UTC [32137] LOG: database system was shut down at 2026-08-08 23:42:01 UTC
2026-08-08 23:42:02.451 UTC [32135] LOG: database system is ready to accept connections
I20260808 23:42:03.486662 19178 mini_postgres.cc:96] Postgres bound to 37209
2026-08-08 23:42:03.508 UTC [32147] FATAL: database "slave" does not exist
127.18.186.148:37209 - accepting connections
I20260808 23:42:03.509223 19178 mini_ranger.cc:161] Starting Ranger...
I20260808 23:42:03.537827 19178 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 23:42:04.017620 19178 mini_ranger.cc:87] Created ranger Postgres database
I20260808 23:42:04.017732 19178 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-admin
2026-08-08 23:42:04,306 [I] DB FLAVOR :POSTGRES
2026-08-08 23:42:04,306 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:42:04,307 [I] Checking connection..
2026-08-08 23:42:04,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 23:42:04,559 [I] Checking connection passed.
2026-08-08 23:42:04,559 [I] --------- Verifying version history table ---------
2026-08-08 23:42:04,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:42:04,831 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:42:04,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:42:05,087 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:42:05,088 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 23:42:05,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 23:42:05.355 UTC [32220] WARNING: there is no transaction in progress
2026-08-08 23:42:05,363 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 23:42:05,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:42:05,617 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 23:42:05,617 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:42:05,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 23:42:05,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 23:42:06,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5k54.c.gcp-upstream.internal','N') ;"
2026-08-08 23:42:06,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:42:06,654 [I] Table x_portal_user does not exist in database ranger
2026-08-08 23:42:06,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:42:06,895 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 23:42:06,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:42:07,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:42:07,399 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 23:42:07,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 23:42:08.354 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.371 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.388 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.403 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.421 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.442 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.499 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.507 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.522 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.533 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.543 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.557 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.567 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.573 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.581 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:08.587 UTC [32335] WARNING: there is no transaction in progress
2026-08-08 23:42:09,224 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 23:42:09,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 23:42:09,449 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 23:42:09,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:42:09,690 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 23:42:09,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:42:09,952 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 23:42:09,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:42:10,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:42:10,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5k54.c.gcp-upstream.internal';"
2026-08-08 23:42:10,704 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 23:42:10,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 23:42:10,974 [I] DB_PATCHES have already been applied
I20260808 23:42:10.980736 19178 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-admin:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskX1Oigb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 23:42:10.980829 19178 mini_ranger.cc:193] Using host: 127.18.186.148
I20260808 23:42:11.000452 19178 mini_ranger.cc:227] Ranger admin URL: http://127.18.186.148:43737
Aug 08, 2026 11:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 23:42:12.189693 19178 mini_ranger.cc:160] Time spent starting Ranger: real 8.680s user 0.000s sys 0.053s
Aug 08, 2026 11:42:31 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:42:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 11:42:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:42:31 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:42:32 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 23:42:33.224462 19178 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.186.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.186.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/rangerkms_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.186.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/rangerkms_127.18.186.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.186.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.186.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.186.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/HTTP_127.18.186.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.186.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/HTTP_127.18.186.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 23:42:33 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232553, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 23:42:33.428011 19178 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 23:42:33.453033 19178 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 23:42:34.363979 19178 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 23:42:34.364084 19178 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms
I20260808 23:42:34.364114 19178 mini_ranger_kms.cc:227] Using postgres at 127.18.186.148:37209
2026-08-08 23:42:34,738 [I] DB FLAVOR :POSTGRES
2026-08-08 23:42:34,739 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:42:34,739 [I] Checking connection
2026-08-08 23:42:34,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 23:42:34,994 [I] connection success
2026-08-08 23:42:34,995 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 23:42:34,995 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 23:42:34,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 23:42:35,224 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 23:42:35,225 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:42:35,225 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 23:42:35,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.186.148:37209/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 23:42:35,482 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 23:42:35.949594 19178 mini_ranger_kms.cc:310] Created kms service
I20260808 23:42:36.141846 19178 mini_ranger_kms.cc:326] Created kudu user
I20260808 23:42:36.187923 19178 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 23:42:36 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232553, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:36.603982 19178 mini_ranger_kms.cc:384] Added ranger policy
I20260808 23:42:36.604364 19178 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms:/tmp/dist-test-taskX1Oigb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskX1Oigb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 23:42:36.604449 19178 mini_ranger_kms.cc:240] Using host: 127.18.186.148
I20260808 23:42:36.624719 19178 mini_ranger_kms.cc:276] Ranger KMS PID: 32542
I20260808 23:42:36.624892 19178 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.186.148:41805
Aug 08, 2026 11:42:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:42:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:42:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:42:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:42:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:42:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 23:42:38 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786232558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.186.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:39.064345 19178 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.636s user 0.065s sys 0.339s
I20260808 23:42:39.064613 19178 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 23:42:39.064867 19178 mini_ranger_kms.cc:401] 127.18.186.148:41805/kms/v1/keys
Aug 08 23:42:39 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.186.148@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
Aug 08 23:42:39 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.186.148@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
Aug 08 23:42:41 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:42:42.017515 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:37209
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:37209
--ranger_config_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:41805/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:42:42.351816 32601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:42.352144 32601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:42.352309 32601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:42.359344 32601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:42:42.359511 32601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:42.359625 32601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:42.359732 32601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:42:42.359826 32601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:42:42.369655 32601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.186.148:41805/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:37209
--ranger_config_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:37209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:42.370991 32601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:42.372468 32601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:42.384531 32608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:42.384708 32611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:42.384703 32609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:42.385155 32601 server_base.cc:1034] running on GCE node
I20260808 23:42:42.386080 32601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:42.387504 32601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:42.388726 32601 hybrid_clock.cc:648] HybridClock initialized: now 1786232562388697 us; error 39 us; skew 500 ppm
I20260808 23:42:42.389075 32601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:42 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232562, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:42.395545 32601 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:42:42.396974 32601 webserver.cc:460] Webserver started at http://127.18.186.190:41207/ using document root <none> and password file <none>
I20260808 23:42:42.397603 32601 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:42.397709 32601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:42.397949 32601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:42:42 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232562, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:42.456595 32601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "65d4dad255e348bf869b769fa27bc2e4"
format_stamp: "Formatted at 2026-08-08 23:42:42 on dist-test-slave-5k54"
server_key: "6cf9d1e53c5327c4cf9997ef7d2e21a4"
server_key_iv: "4c795f28583d207ffb6426cdad6304b0"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:42.457477 32601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "65d4dad255e348bf869b769fa27bc2e4"
format_stamp: "Formatted at 2026-08-08 23:42:42 on dist-test-slave-5k54"
server_key: "6cf9d1e53c5327c4cf9997ef7d2e21a4"
server_key_iv: "4c795f28583d207ffb6426cdad6304b0"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:42.462899 32601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 23:42:42.479796 32620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:42.481607 32601 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:42:42.481806 32601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "65d4dad255e348bf869b769fa27bc2e4"
format_stamp: "Formatted at 2026-08-08 23:42:42 on dist-test-slave-5k54"
server_key: "6cf9d1e53c5327c4cf9997ef7d2e21a4"
server_key_iv: "4c795f28583d207ffb6426cdad6304b0"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:42.482069 32601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:42.497786 32601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:42.501701 32601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:42.502039 32601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:42.521212 32601 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:37209
I20260808 23:42:42.521237 32682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:37209 every 8 connection(s)
I20260808 23:42:42.522857 32601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:42:42.526386 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 32601
I20260808 23:42:42.526636 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:42:42.526969 32683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:42.527554 19178 external_mini_cluster.cc:1428] Setting key 46d3fbcf16790deee5b3bdc557040b8e
I20260808 23:42:42.541622 32683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Bootstrap starting.
Aug 08 23:42:42 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:42.544976 32683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:42.546000 32683 log.cc:824] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:42.548780 32683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: No bootstrap required, opened a new log
I20260808 23:42:42.550670 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:42.529619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.18.186.190:37209)
0808 23:42:42.530513 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:42.530524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:42.530575 (+ 51us) server_negotiation.cc:407] Connection header received
0808 23:42:42.532217 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:42.532240 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:42.532633 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:42.532978 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:42.534637 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:42.536058 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:42.537888 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:42.538588 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:42.538725 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:42.543678 (+ 4953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:42.543686 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:42.543700 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:42.543739 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:42.546859 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:42.547561 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:42.547573 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:42.547579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:42.547729 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:42.548111 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:42.548118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:42.548121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:42.548587 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:42.548904 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:42.549353 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 23:42:42.549725 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":445,"threads_started":1}
I20260808 23:42:42.556521 32683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } }
I20260808 23:42:42.556869 32683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:42.556986 32683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d4dad255e348bf869b769fa27bc2e4, State: Initialized, Role: FOLLOWER
I20260808 23:42:42.557508 32683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } }
I20260808 23:42:42.557758 32683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:42:42.557915 32683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:42:42.558087 32683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:42.560564 32683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } }
I20260808 23:42:42.561084 32683 leader_election.cc:302] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d4dad255e348bf869b769fa27bc2e4; no voters:
I20260808 23:42:42.561625 32683 leader_election.cc:288] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:42:42.561755 32691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:42:42.563009 32691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 1 LEADER]: Becoming Leader. State: Replica: 65d4dad255e348bf869b769fa27bc2e4, State: Running, Role: LEADER
I20260808 23:42:42.563601 32691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } }
I20260808 23:42:42.563800 32683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:42:42.566756 32692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65d4dad255e348bf869b769fa27bc2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } } }
I20260808 23:42:42.566893 32693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d4dad255e348bf869b769fa27bc2e4. Latest consensus state: current_term: 1 leader_uuid: "65d4dad255e348bf869b769fa27bc2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } } }
I20260808 23:42:42.567210 32692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:42.567320 32693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:42.572476 32699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:42:42.578962 32699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:42:42.579583 32683 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 23:42:42.597675 32699 catalog_manager.cc:1347] Generated new cluster ID: 877474afb1d14f5d9f1988ca079cb4d8
I20260808 23:42:42.597882 32699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:42:42.628628 32699 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:42:42.630177 32699 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:42:42.642189 32699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Generated new TSK 0
I20260808 23:42:42.643050 32699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:42:43.453656 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:41805/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37209
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:42:43.534765 32679 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.18.186.190:37209
Aug 08 23:42:43 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786232563, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 23:42:43.890774 32736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:43.891187 32736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:43.891415 32736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:43.903120 32736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:43.903335 32736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:43.903606 32736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:42:43.919584 32736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.186.148:41805/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:43.921566 32736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:43.923285 32736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:43.946973 32747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:43.947065 32748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:43.947683 32751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:43.948372 32736 server_base.cc:1034] running on GCE node
I20260808 23:42:43.949090 32736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:43.950196 32736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:43.951465 32736 hybrid_clock.cc:648] HybridClock initialized: now 1786232563951365 us; error 48 us; skew 500 ppm
I20260808 23:42:43.951815 32736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:43 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232563, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:43.955508 32736 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:42:43.957325 32736 webserver.cc:460] Webserver started at http://127.18.186.129:35351/ using document root <none> and password file <none>
I20260808 23:42:43.958250 32736 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:43.958395 32736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:43.958778 32736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:42:43 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232563, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:43.990103 32736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "39288e833d934b91aae1535e718dd3fe"
format_stamp: "Formatted at 2026-08-08 23:42:43 on dist-test-slave-5k54"
server_key: "37defb22659e8720edfde08a55f729a4"
server_key_iv: "c2dc13b8cb46ac771d720874b1c2e642"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:43.991346 32736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "39288e833d934b91aae1535e718dd3fe"
format_stamp: "Formatted at 2026-08-08 23:42:43 on dist-test-slave-5k54"
server_key: "37defb22659e8720edfde08a55f729a4"
server_key_iv: "c2dc13b8cb46ac771d720874b1c2e642"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:43.998983 32736 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260808 23:42:44.027765 32758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:44.029606 32736 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.005s
I20260808 23:42:44.029934 32736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "39288e833d934b91aae1535e718dd3fe"
format_stamp: "Formatted at 2026-08-08 23:42:43 on dist-test-slave-5k54"
server_key: "37defb22659e8720edfde08a55f729a4"
server_key_iv: "c2dc13b8cb46ac771d720874b1c2e642"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:44.030184 32736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:44.054706 32736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:44.060024 32736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:44.060436 32736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:44.063112 32736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:42:44.076313 32736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:42:44.076463 32736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:44.076611 32736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:42:44.076695 32736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:42:44 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232563, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:44.280457 32736 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:32933
I20260808 23:42:44.280499 411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:32933 every 8 connection(s)
I20260808 23:42:44.282299 32736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:42:44.291700 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 32736
I20260808 23:42:44.291931 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:42:44.292760 19178 external_mini_cluster.cc:1428] Setting key 1df4d1084fb4ad0ac7d7caa07fdd038e
Aug 08 23:42:44 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232563, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:44.324601 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:44.286516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:45499 (local address 127.18.186.190:37209)
0808 23:42:44.287655 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:44.287661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:44.298436 (+ 10775us) server_negotiation.cc:407] Connection header received
0808 23:42:44.301229 (+ 2793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:44.301237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:44.301344 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:44.301465 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:44.305492 (+ 4027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:44.306655 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:44.309095 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:44.309738 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:44.309839 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:44.316616 (+ 6777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:44.316623 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:44.316628 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:44.316668 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:44.320492 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:44.321594 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:44.321600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:44.321605 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:44.321769 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:44.322372 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:44.322376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:44.322378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:44.322664 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:44.322780 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:44.323835 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0808 23:42:44.324093 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":877,"threads_started":1}
I20260808 23:42:44.325984 414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:44.285802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37209 (local address 127.18.186.129:45499)
0808 23:42:44.298261 (+ 12459us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:44.298324 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:44.300853 (+ 2529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:44.303773 (+ 2920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:44.303786 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:44.304448 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:44.305279 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:44.305302 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:44.306854 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:44.306859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:44.308915 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:44.308945 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:44.310210 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:44.310216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:44.310502 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:44.313445 (+ 2943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:44.313473 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:44.316324 (+ 2851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:44.320689 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:44.320702 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:44.320722 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:44.321441 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:44.321941 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:44.321946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:44.321950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:44.322223 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:44.322814 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:44.322832 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:44.323330 (+ 498us) client_negotiation.cc:769] Sending connection context
0808 23:42:44.323709 (+ 379us) client_negotiation.cc:240] Negotiation successful
0808 23:42:44.324344 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12112,"thread_start_us":15342,"threads_started":1}
I20260808 23:42:44.327903 412 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37209
I20260808 23:42:44.328224 412 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:44.329094 412 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
I20260808 23:42:44.331773 32638 ts_manager.cc:194] Registered new tserver with Master: 39288e833d934b91aae1535e718dd3fe (127.18.186.129:32933)
I20260808 23:42:44.334002 32638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:45499
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:42:44.376449 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:41805/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37209
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:42:44 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232563, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
W20260808 23:42:44.860090 418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:44.860567 418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:44.860752 418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:44.872485 418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:44.872704 418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:44.872941 418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:42:44.891009 418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.186.148:41805/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:44.892701 418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:44.894434 418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:44.910429 435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:44.911417 437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:44.928720 440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:44.929078 418 server_base.cc:1034] running on GCE node
I20260808 23:42:44.929804 418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:44.930968 418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:44.932246 418 hybrid_clock.cc:648] HybridClock initialized: now 1786232564932177 us; error 65 us; skew 500 ppm
I20260808 23:42:44.932585 418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:44 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232564, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:44.936338 418 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:42:44.938230 418 webserver.cc:460] Webserver started at http://127.18.186.130:44471/ using document root <none> and password file <none>
I20260808 23:42:44.938982 418 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:44.939128 418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:44.939508 418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:42:44 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232564, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:44.963471 418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "944fd755d8f1403d8c2ee55229f69295"
format_stamp: "Formatted at 2026-08-08 23:42:44 on dist-test-slave-5k54"
server_key: "4ae0fa7a7c871952efcd7e123e286437"
server_key_iv: "36abfce36a7315eed66cb9e61a3e8c84"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:44.964504 418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "944fd755d8f1403d8c2ee55229f69295"
format_stamp: "Formatted at 2026-08-08 23:42:44 on dist-test-slave-5k54"
server_key: "4ae0fa7a7c871952efcd7e123e286437"
server_key_iv: "36abfce36a7315eed66cb9e61a3e8c84"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:44.970969 418 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260808 23:42:44.992502 447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:44.994208 418 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260808 23:42:44.994437 418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "944fd755d8f1403d8c2ee55229f69295"
format_stamp: "Formatted at 2026-08-08 23:42:44 on dist-test-slave-5k54"
server_key: "4ae0fa7a7c871952efcd7e123e286437"
server_key_iv: "36abfce36a7315eed66cb9e61a3e8c84"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:44.994735 418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:45.019699 418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:45.025405 418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:45.025735 418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:45.026842 418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:42:45.028467 418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:42:45.028576 418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:45.028728 418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:42:45.028801 418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:45.071399 418 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:34933
I20260808 23:42:45.071470 566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:34933 every 8 connection(s)
I20260808 23:42:45.072882 418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:42:45.083134 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 418
I20260808 23:42:45.083359 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:42:45.083873 19178 external_mini_cluster.cc:1428] Setting key 60cad05056ad3378c5e7543814024e1d
Aug 08 23:42:45 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232564, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:45.097944 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:45.076144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:39549 (local address 127.18.186.190:37209)
0808 23:42:45.076684 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:45.076691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:45.077893 (+ 1202us) server_negotiation.cc:407] Connection header received
0808 23:42:45.079880 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:45.079885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:45.079984 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:45.080110 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:45.081523 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.082617 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.085041 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.085683 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.085835 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:45.090117 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:45.090127 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:45.090135 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:45.090181 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:45.094091 (+ 3910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.094980 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.094988 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.094994 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.095157 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.095716 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.095723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.095729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.096190 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:45.096346 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:45.097202 (+ 856us) server_negotiation.cc:299] Negotiation successful
0808 23:42:45.097482 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":290,"threads_started":1}
I20260808 23:42:45.099095 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:45.076876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37209 (local address 127.18.186.130:39549)
0808 23:42:45.077713 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:45.077759 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:45.079591 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:45.080316 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:45.080323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:45.080757 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:45.081347 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:45.081364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.082827 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.082832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:45.084908 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:45.084923 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.085957 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.085965 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:45.086239 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:45.087021 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:45.087050 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:45.089871 (+ 2821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:45.094256 (+ 4385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:45.094265 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:45.094280 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:45.094826 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:45.095269 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:45.095274 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:45.095277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:45.095573 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:45.096337 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:45.096344 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:45.096750 (+ 406us) client_negotiation.cc:769] Sending connection context
0808 23:42:45.097044 (+ 294us) client_negotiation.cc:240] Negotiation successful
0808 23:42:45.097550 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":344,"threads_started":1}
I20260808 23:42:45.100801 567 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37209
I20260808 23:42:45.101140 567 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:45.101974 567 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
I20260808 23:42:45.103734 32638 ts_manager.cc:194] Registered new tserver with Master: 944fd755d8f1403d8c2ee55229f69295 (127.18.186.130:34933)
I20260808 23:42:45.104837 32638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:39549
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:42:45.147364 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:41805/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37209
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:42:45.338198 412 heartbeater.cc:503] Master 127.18.186.190:37209 was elected leader, sending a full tablet report...
W20260808 23:42:45.480501 575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:45.480855 575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:45.480957 575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:45.488312 575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:45.488415 575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:45.488543 575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:42:45.498332 575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.186.148:41805/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:37209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:45.499624 575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:45.500994 575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:45.514315 581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:45.514431 585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:45.514173 582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:45.515547 575 server_base.cc:1034] running on GCE node
I20260808 23:42:45.516176 575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:45.517153 575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:45.518393 575 hybrid_clock.cc:648] HybridClock initialized: now 1786232565518345 us; error 53 us; skew 500 ppm
I20260808 23:42:45.518718 575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:45 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232565, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:45.521941 575 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:42:45.523492 575 webserver.cc:460] Webserver started at http://127.18.186.131:41605/ using document root <none> and password file <none>
I20260808 23:42:45.524175 575 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:45.524300 575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:45.524641 575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:42:45 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232565, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:45.538658 575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0"
format_stamp: "Formatted at 2026-08-08 23:42:45 on dist-test-slave-5k54"
server_key: "a8e709cf4cb5822833dc0efda6869029"
server_key_iv: "2dfb543176a7a950a287bd762222b83d"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:45.539618 575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0"
format_stamp: "Formatted at 2026-08-08 23:42:45 on dist-test-slave-5k54"
server_key: "a8e709cf4cb5822833dc0efda6869029"
server_key_iv: "2dfb543176a7a950a287bd762222b83d"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:45.545077 575 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 23:42:45.562286 592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:45.563951 575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 23:42:45.564124 575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0"
format_stamp: "Formatted at 2026-08-08 23:42:45 on dist-test-slave-5k54"
server_key: "a8e709cf4cb5822833dc0efda6869029"
server_key_iv: "2dfb543176a7a950a287bd762222b83d"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:45.564355 575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:45.576061 575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:45.580180 575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:45.580480 575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:45.581492 575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:42:45.582947 575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:42:45.583017 575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:45.583138 575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:42:45.583187 575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:45.620050 575 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:44915
I20260808 23:42:45.620086 710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:44915 every 8 connection(s)
I20260808 23:42:45.621374 575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:42:45.623391 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 575
I20260808 23:42:45.623633 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:42:45.624192 19178 external_mini_cluster.cc:1428] Setting key 82cd23e5669fa80219f624d78cacba03
Aug 08 23:42:45 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232565, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:45.641238 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:45.624056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:57791 (local address 127.18.186.190:37209)
0808 23:42:45.624630 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:45.624637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:45.625629 (+ 992us) server_negotiation.cc:407] Connection header received
0808 23:42:45.627912 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:45.627918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:45.627999 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:45.628117 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:45.629483 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.630181 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.631771 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.632228 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.632363 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:45.635828 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:45.635833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:45.635838 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:45.635891 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:45.638202 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.638892 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.638900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.638904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.639031 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.639481 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.639485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.639489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.639819 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:45.639974 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:45.640642 (+ 668us) server_negotiation.cc:299] Negotiation successful
0808 23:42:45.640859 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":256,"threads_started":1}
I20260808 23:42:45.642200 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:45.624607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37209 (local address 127.18.186.131:57791)
0808 23:42:45.625450 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:45.625509 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:45.627648 (+ 2139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:45.628359 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:45.628367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:45.628761 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:45.629318 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:45.629337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.630326 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.630331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:45.631625 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:45.631636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.632448 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.632455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:45.632699 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:45.633399 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:45.633419 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:45.635640 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:45.638352 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:45.638360 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:45.638375 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:45.638760 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:45.639140 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:45.639145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:45.639149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:45.639342 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:45.639942 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:45.639949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:45.640259 (+ 310us) client_negotiation.cc:769] Sending connection context
0808 23:42:45.640535 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 23:42:45.641009 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":340,"threads_started":1}
I20260808 23:42:45.643594 711 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37209
I20260808 23:42:45.643843 711 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:45.644476 711 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
I20260808 23:42:45.645781 32638 ts_manager.cc:194] Registered new tserver with Master: 5a6df6b2f44f4d95afbc0f9f0e2fabc0 (127.18.186.131:44915)
I20260808 23:42:45.646539 32638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:57791
I20260808 23:42:45.651775 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:42:45.653422 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 32736
Aug 08 23:42:45 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:42:45.669978 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:45.654345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36490 (local address 127.18.186.129:32933)
0808 23:42:45.654970 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:45.654976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:45.654999 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:42:45.655052 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:45.655058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:45.655231 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:45.655352 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:45.656789 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.657756 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.659415 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.660006 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.660101 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:45.664244 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:45.664255 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:45.664267 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:45.664302 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:45.667246 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.668099 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.668106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.668109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.668257 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.668662 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.668665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.668667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.668981 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:45.669252 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:45.669506 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 23:42:45.669696 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":301,"threads_started":1}
I20260808 23:42:45.671182 407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:45.754194 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 32736
I20260808 23:42:45.772327 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 418
Aug 08 23:42:45 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:42:45.791093 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:45.773241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.18.186.130:34933)
0808 23:42:45.774006 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:45.774013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:45.774037 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:42:45.774096 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:45.774102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:45.774379 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:45.774548 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:45.776900 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.778106 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.779590 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.780166 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.780312 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:45.785318 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:45.785330 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:45.785343 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:45.785377 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:45.788176 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.788910 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.788917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.788920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.789051 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.789458 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.789461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.789463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.789802 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:45.790128 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:45.790471 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 23:42:45.790740 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":476,"threads_started":1}
I20260808 23:42:45.792586 562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:45.875942 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 418
I20260808 23:42:45.894006 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 575
Aug 08 23:42:45 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:42:45.910537 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:45.895026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.18.186.131:44915)
0808 23:42:45.895760 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:45.895773 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:45.895804 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:42:45.895864 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:45.895870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:45.896076 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:45.896212 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:45.897569 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.898578 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.899945 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:45.900488 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:45.900585 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:45.904941 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:45.904952 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:45.904965 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:45.904998 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:45.907996 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.908650 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.908657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.908659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.908839 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:45.909165 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:45.909169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:45.909171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:45.909502 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:45.909736 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:45.910068 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 23:42:45.910250 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":440,"threads_started":1}
I20260808 23:42:45.911839 706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:45.994100 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 575
I20260808 23:42:46.010722 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 32601
I20260808 23:42:46.012048 32678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:46.119196 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 32601
I20260808 23:42:46.143051 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:37209
--webserver_interface=127.18.186.190
--webserver_port=41207
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:37209
--ranger_config_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:41805/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.186.148:41805/kms with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:42:46.463148 724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:46.463440 724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:46.463627 724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:46.470698 724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:42:46.470798 724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:46.470865 724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:46.470973 724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:42:46.471048 724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:42:46.481119 724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.186.148:41805/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:37209
--ranger_config_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:37209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=41207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:46.482438 724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:46.483889 724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:46.496266 731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:46.496309 734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:46.496330 729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:46.497021 724 server_base.cc:1034] running on GCE node
I20260808 23:42:46.497789 724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:46.499225 724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:46.500448 724 hybrid_clock.cc:648] HybridClock initialized: now 1786232566500384 us; error 38 us; skew 500 ppm
I20260808 23:42:46.500770 724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:46 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232566, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:46.504191 724 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:42:46.505785 724 webserver.cc:460] Webserver started at http://127.18.186.190:41207/ using document root <none> and password file <none>
I20260808 23:42:46.506538 724 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:46.506675 724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:42:46 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232566, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:46.590987 724 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:42:46.594571 742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:46.595976 724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:42:46.596177 724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "65d4dad255e348bf869b769fa27bc2e4"
format_stamp: "Formatted at 2026-08-08 23:42:42 on dist-test-slave-5k54"
server_key: "6cf9d1e53c5327c4cf9997ef7d2e21a4"
server_key_iv: "4c795f28583d207ffb6426cdad6304b0"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:46.597151 724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:46.616856 724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:46.621059 724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:46.621353 724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:46.639837 724 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:37209
I20260808 23:42:46.639850 796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:37209 every 8 connection(s)
I20260808 23:42:46.641364 724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:42:46.648277 798 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:42:46.649360 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 724
I20260808 23:42:46.650513 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:41805/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:32933
--local_ip_for_outbound_sockets=127.18.186.129
--tserver_master_addrs=127.18.186.190:37209
--webserver_port=35351
--webserver_interface=127.18.186.129
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.186.148:41805/kms with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:42:46.653223 798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Bootstrap starting.
I20260808 23:42:46.674463 798 log.cc:824] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:46.684134 798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:42:46.684689 798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Bootstrap complete.
I20260808 23:42:46.695899 798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } }
I20260808 23:42:46.696895 798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d4dad255e348bf869b769fa27bc2e4, State: Initialized, Role: FOLLOWER
I20260808 23:42:46.697625 798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } }
I20260808 23:42:46.697898 798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:42:46.698050 798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:42:46.698268 798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:42:46.702976 798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } }
I20260808 23:42:46.703590 798 leader_election.cc:302] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d4dad255e348bf869b769fa27bc2e4; no voters:
I20260808 23:42:46.704391 798 leader_election.cc:288] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:42:46.704537 802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:42:46.705817 802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [term 2 LEADER]: Becoming Leader. State: Replica: 65d4dad255e348bf869b769fa27bc2e4, State: Running, Role: LEADER
I20260808 23:42:46.706487 802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } }
I20260808 23:42:46.706884 798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:42:46.709349 804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d4dad255e348bf869b769fa27bc2e4. Latest consensus state: current_term: 2 leader_uuid: "65d4dad255e348bf869b769fa27bc2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } } }
I20260808 23:42:46.709844 804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:46.710507 803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "65d4dad255e348bf869b769fa27bc2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d4dad255e348bf869b769fa27bc2e4" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37209 } } }
I20260808 23:42:46.710917 803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4 [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:46.714330 808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:42:46.722011 808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:42:46.724100 808 catalog_manager.cc:1259] Loaded cluster ID: 877474afb1d14f5d9f1988ca079cb4d8
I20260808 23:42:46.724325 808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:42:46.730211 808 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:42:46.732414 808 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Loaded TSK: 0
I20260808 23:42:46.734560 808 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:42:46.743847 798 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 23:42:47.115475 800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:47.115988 800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:47.116189 800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:47.128000 800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:47.128142 800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:47.128369 800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:42:47.144205 800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.186.148:41805/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:32933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=35351
--tserver_master_addrs=127.18.186.190:37209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:47.146116 800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:47.147939 800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:47.165386 839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:47.165961 840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:47.171253 800 server_base.cc:1034] running on GCE node
W20260808 23:42:47.172034 842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:47.173317 800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:47.180815 800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:47.182103 800 hybrid_clock.cc:648] HybridClock initialized: now 1786232567182037 us; error 61 us; skew 500 ppm
I20260808 23:42:47.182547 800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:47 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:47.190747 800 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:42:47.220074 800 webserver.cc:460] Webserver started at http://127.18.186.129:35351/ using document root <none> and password file <none>
I20260808 23:42:47.221293 800 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:47.221561 800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:42:47 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:47.273121 800 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.010s sys 0.005s
I20260808 23:42:47.285765 852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:47.287933 800 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.003s sys 0.008s
I20260808 23:42:47.288196 800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "39288e833d934b91aae1535e718dd3fe"
format_stamp: "Formatted at 2026-08-08 23:42:43 on dist-test-slave-5k54"
server_key: "37defb22659e8720edfde08a55f729a4"
server_key_iv: "c2dc13b8cb46ac771d720874b1c2e642"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:47.289804 800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:47.325299 800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:47.330366 800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:47.330677 800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:47.331875 800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:42:47.335041 800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:42:47.335170 800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:47.335302 800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:42:47.335400 800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:47.448400 800 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:32933
I20260808 23:42:47.448457 979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:32933 every 8 connection(s)
I20260808 23:42:47.450117 800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:42:47.453914 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 800
I20260808 23:42:47.454804 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:41805/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:34933
--local_ip_for_outbound_sockets=127.18.186.130
--tserver_master_addrs=127.18.186.190:37209
--webserver_port=44471
--webserver_interface=127.18.186.130
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.186.148:41805/kms with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:42:47 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:47.502092 983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:47.454634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37209 (local address 127.18.186.129:57923)
0808 23:42:47.466199 (+ 11565us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:47.466289 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:47.469827 (+ 3538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:47.473413 (+ 3586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:47.473424 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:47.474019 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:47.474597 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:47.474615 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:47.476907 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:47.476916 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:47.479228 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:47.479248 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:47.481240 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:47.481249 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:47.481577 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:47.487531 (+ 5954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:47.487582 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:47.490743 (+ 3161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:47.496485 (+ 5742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:47.496497 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:47.496520 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:47.497264 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:47.497921 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:47.497929 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:47.497935 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:47.498233 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:47.499329 (+ 1096us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:47.499338 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:47.499815 (+ 477us) client_negotiation.cc:769] Sending connection context
0808 23:42:47.500125 (+ 310us) client_negotiation.cc:240] Negotiation successful
0808 23:42:47.500697 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11076,"thread_start_us":15726,"threads_started":1}
I20260808 23:42:47.502287 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:47.454553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:57923 (local address 127.18.186.190:37209)
0808 23:42:47.471982 (+ 17429us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:47.471999 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:47.472042 (+ 43us) server_negotiation.cc:407] Connection header received
0808 23:42:47.472423 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:47.472438 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:47.472828 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:47.473245 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:47.474827 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:47.476687 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:47.479418 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:47.480608 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:47.480819 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:47.491189 (+ 10370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:47.491205 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:47.491233 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:47.491302 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:47.496227 (+ 4925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:47.497481 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:47.497495 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:47.497506 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:47.497743 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:47.498385 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:47.498391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:47.498395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:47.499158 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:47.499392 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:47.500409 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0808 23:42:47.500926 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17153,"thread_start_us":16930,"threads_started":1}
I20260808 23:42:47.506260 980 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37209
I20260808 23:42:47.506611 980 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:47.507680 980 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
W20260808 23:42:47.511184 980 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:37209 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 23:42:47.511463 980 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:47.511771 980 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
I20260808 23:42:47.513118 980 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:47.513422 980 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
W20260808 23:42:47.514629 980 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:42:47.657294 793 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.18.186.190:37209
W20260808 23:42:47.886545 984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:47.886988 984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:47.887184 984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:47.899003 984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:47.899212 984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:47.899473 984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:42:47.914796 984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.186.148:41805/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:34933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=44471
--tserver_master_addrs=127.18.186.190:37209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:47.916414 984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:47.917758 984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:47.935488 1001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:47.935623 1002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:47.943208 1004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:47.943773 984 server_base.cc:1034] running on GCE node
I20260808 23:42:47.944414 984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:47.947724 984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:47.948987 984 hybrid_clock.cc:648] HybridClock initialized: now 1786232567948924 us; error 59 us; skew 500 ppm
I20260808 23:42:47.949322 984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:47 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:47.952839 984 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:42:47.954677 984 webserver.cc:460] Webserver started at http://127.18.186.130:44471/ using document root <none> and password file <none>
I20260808 23:42:47.955403 984 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:47.955600 984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:42:47 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
Aug 08 23:42:47 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:47.983170 984 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 23:42:47.996054 1012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:48.000924 984 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.002s sys 0.010s
I20260808 23:42:48.001224 984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "944fd755d8f1403d8c2ee55229f69295"
format_stamp: "Formatted at 2026-08-08 23:42:44 on dist-test-slave-5k54"
server_key: "4ae0fa7a7c871952efcd7e123e286437"
server_key_iv: "36abfce36a7315eed66cb9e61a3e8c84"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:48.002772 984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:48.029109 984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:48.033943 984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:48.034227 984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:48.035346 984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:42:48.036860 984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:42:48.036985 984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:48.037114 984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:42:48.037199 984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:48.081269 984 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:34933
I20260808 23:42:48.081333 1138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:34933 every 8 connection(s)
I20260808 23:42:48.082859 984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:42:48.090368 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 984
I20260808 23:42:48.091205 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.186.148:41805/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:44915
--local_ip_for_outbound_sockets=127.18.186.131
--tserver_master_addrs=127.18.186.190:37209
--webserver_port=41605
--webserver_interface=127.18.186.131
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.186.148:41805/kms with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:42:48 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:48.131126 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:48.086924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37209 (local address 127.18.186.130:33563)
0808 23:42:48.100106 (+ 13182us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:48.100179 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:48.102533 (+ 2354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:48.103479 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:48.103492 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:48.104162 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:48.105070 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:48.105098 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.106792 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.106800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:48.109493 (+ 2693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:48.109511 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.110621 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.110631 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:48.110955 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:48.111881 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:48.111922 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:48.115011 (+ 3089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:48.126223 (+ 11212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:48.126233 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:48.126252 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:48.126871 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:48.127332 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:48.127337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:48.127340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:48.127616 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:48.128274 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:48.128283 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:48.128719 (+ 436us) client_negotiation.cc:769] Sending connection context
0808 23:42:48.128995 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 23:42:48.129555 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12812,"thread_start_us":12552,"threads_started":1}
I20260808 23:42:48.133065 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:48.087730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:33563 (local address 127.18.186.190:37209)
0808 23:42:48.088876 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:48.088886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:48.100319 (+ 11433us) server_negotiation.cc:407] Connection header received
0808 23:42:48.102851 (+ 2532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:48.102856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:48.102986 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:48.103112 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:48.105293 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.106548 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.109672 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.110354 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.110509 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:48.115376 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:48.115386 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:48.115395 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:48.115478 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:48.126038 (+ 10560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:48.127036 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:48.127041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:48.127045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:48.127194 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:48.127769 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:48.127774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:48.127777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:48.128130 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:48.132295 (+ 4165us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:48.132442 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 23:42:48.132676 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":856,"threads_started":1}
I20260808 23:42:48.135025 1139 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37209
I20260808 23:42:48.135349 1139 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:48.136335 1139 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
I20260808 23:42:48.138501 759 ts_manager.cc:194] Registered new tserver with Master: 944fd755d8f1403d8c2ee55229f69295 (127.18.186.130:34933)
I20260808 23:42:48.140668 759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:33563
Aug 08 23:42:48 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:48.514770 980 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:48.515187 980 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
I20260808 23:42:48.516513 759 ts_manager.cc:194] Registered new tserver with Master: 39288e833d934b91aae1535e718dd3fe (127.18.186.129:32933)
I20260808 23:42:48.517755 759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:57923
W20260808 23:42:48.565506 1143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:48.565847 1143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:48.566059 1143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:48.577682 1143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:48.577929 1143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:48.578171 1143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:42:48.594149 1143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.186.148:41805/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:44915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=41605
--tserver_master_addrs=127.18.186.190:37209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:48.596113 1143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:48.597875 1143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:48.636849 1148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:48.648335 1143 server_base.cc:1034] running on GCE node
W20260808 23:42:48.651219 1152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:48.651661 1149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:48.652745 1143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:48.661885 1143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:48.663164 1143 hybrid_clock.cc:648] HybridClock initialized: now 1786232568663090 us; error 77 us; skew 500 ppm
I20260808 23:42:48.663633 1143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:48 dist-test-slave-5k54 krb5kdc[32093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:48.668347 1143 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:42:48.688917 1143 webserver.cc:460] Webserver started at http://127.18.186.131:41605/ using document root <none> and password file <none>
I20260808 23:42:48.690096 1143 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:48.690327 1143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:42:48 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:48.749775 1143 fs_manager.cc:709] Time spent opening directory manager: real 0.023s user 0.015s sys 0.004s
I20260808 23:42:48.760241 1159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:48.771971 1143 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.002s sys 0.004s
I20260808 23:42:48.772334 1143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0"
format_stamp: "Formatted at 2026-08-08 23:42:45 on dist-test-slave-5k54"
server_key: "a8e709cf4cb5822833dc0efda6869029"
server_key_iv: "2dfb543176a7a950a287bd762222b83d"
server_key_version: "kuduclusterkey@0"
I20260808 23:42:48.775544 1143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 23:42:48 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for HTTP/127.18.186.148@KRBTEST.COM
I20260808 23:42:48.812296 1143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:48.820886 1143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:48.821252 1143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:48.823011 1143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:42:48.828759 1143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:42:48.828888 1143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:48.829015 1143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:42:48.829104 1143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:48.950109 1143 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:44915
I20260808 23:42:48.950186 1280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:44915 every 8 connection(s)
I20260808 23:42:48.952335 1143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:42:48.960556 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 1143
Aug 08 23:42:48 dist-test-slave-5k54 krb5kdc[32093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:48.988067 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:48.961555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.18.186.190:37209)
0808 23:42:48.964009 (+ 2454us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:48.964018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:48.964046 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:42:48.964144 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:48.964154 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:48.964277 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:48.967345 (+ 3068us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:48.967587 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.969248 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.972773 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.973572 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.973689 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:48.978398 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:48.978405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:48.978411 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:48.978461 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:48.983065 (+ 4604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:48.984089 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:48.984096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:48.984099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:48.984255 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:48.984824 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:48.984829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:48.984833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:48.985220 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:48.985517 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:48.986991 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0808 23:42:48.987273 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2280,"thread_start_us":494,"threads_started":1}
I20260808 23:42:49.001281 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:48.958540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:43861 (local address 127.18.186.190:37209)
0808 23:42:48.962348 (+ 3808us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:48.962358 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:48.970205 (+ 7847us) server_negotiation.cc:407] Connection header received
0808 23:42:48.973150 (+ 2945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:48.973156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:48.973245 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:48.973343 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:48.976132 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.977642 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.980552 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.981398 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.981523 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:48.986594 (+ 5071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:48.986601 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:48.986608 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:48.986657 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:48.990630 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:48.991644 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:48.991649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:48.991652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:48.991796 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:48.992832 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:48.992838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:48.992842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:48.993208 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:48.993385 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.000486 (+ 7101us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.000761 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3627,"thread_start_us":5105,"threads_started":1}
I20260808 23:42:49.004886 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:48.957680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37209 (local address 127.18.186.131:43861)
0808 23:42:48.970035 (+ 12355us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:48.970096 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:48.972820 (+ 2724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:48.974053 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:48.974068 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:48.974842 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:48.975827 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:48.975856 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.977882 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.977888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:48.980339 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:48.980359 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:48.981929 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:48.981937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:48.982283 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:48.983098 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:48.983137 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:48.986244 (+ 3107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:48.990795 (+ 4551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:48.990804 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:48.990821 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:48.991496 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:48.992383 (+ 887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:48.992388 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:48.992392 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:48.992666 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:48.993363 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:48.993371 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:48.999924 (+ 6553us) client_negotiation.cc:769] Sending connection context
0808 23:42:49.001718 (+ 1794us) client_negotiation.cc:240] Negotiation successful
0808 23:42:49.002375 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12033,"thread_start_us":11768,"threads_started":1}
I20260808 23:42:49.006773 1281 heartbeater.cc:347] Connected to a master server at 127.18.186.190:37209
I20260808 23:42:49.007093 1281 heartbeater.cc:464] Registering TS with master...
I20260808 23:42:49.008076 1281 heartbeater.cc:511] Master 127.18.186.190:37209 requested a full tablet report, sending...
I20260808 23:42:49.009969 759 ts_manager.cc:194] Registered new tserver with Master: 5a6df6b2f44f4d95afbc0f9f0e2fabc0 (127.18.186.131:44915)
I20260808 23:42:49.011126 759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:43861
I20260808 23:42:49.017320 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:42:49.073513 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.053478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51502 (local address 127.18.186.190:37209)
0808 23:42:49.053724 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.053732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.054327 (+ 595us) server_negotiation.cc:407] Connection header received
0808 23:42:49.054657 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.054664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.054759 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.054864 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:49.057474 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.058694 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.060916 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.061610 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.061716 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.065872 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:49.065880 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:49.065886 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:49.065932 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:49.069865 (+ 3933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:49.070837 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:49.070849 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:49.070853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:49.071008 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:49.071562 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:49.071567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:49.071571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:49.072004 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:49.072253 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.072887 (+ 634us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.073143 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 23:42:49.080969 759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:42:49.093218 759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:42:49.145203 1139 heartbeater.cc:503] Master 127.18.186.190:37209 was elected leader, sending a full tablet report...
I20260808 23:42:49.175792 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.133497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.18.186.129:32933)
0808 23:42:49.154076 (+ 20579us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.154092 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.158123 (+ 4031us) server_negotiation.cc:407] Connection header received
0808 23:42:49.158485 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.158495 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.158829 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.159056 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:42:49.160421 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.164656 (+ 4235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.171582 (+ 6926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.173964 (+ 2382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.174203 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.174468 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.175260 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.175460 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20295,"thread_start_us":20591,"threads_started":1}
I20260808 23:42:49.177434 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.132694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:32933 (local address 127.0.0.1:36498)
0808 23:42:49.157965 (+ 25271us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:49.157991 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:49.158246 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:49.159536 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:49.159549 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:49.159604 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:42:49.160168 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.160185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.164903 (+ 4718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.164909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.170295 (+ 5386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.170327 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.174702 (+ 4375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.174708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.174927 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.174995 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 23:42:49.175143 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:42:49.175347 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25093,"thread_start_us":23108,"threads_started":1}
I20260808 23:42:49.183331 1305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.131999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:44915 (local address 127.0.0.1:51792)
0808 23:42:49.155156 (+ 23157us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:49.155228 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:49.155530 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:49.168887 (+ 13357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:49.168895 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:49.168934 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:42:49.169515 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.169532 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.177014 (+ 7482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.177019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.179929 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.179949 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.182086 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.182094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.182298 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.182336 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 23:42:49.182416 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 23:42:49.182550 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22757,"thread_start_us":25632,"threads_started":1}
I20260808 23:42:49.184693 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.158240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:34933 (local address 127.0.0.1:33052)
0808 23:42:49.172209 (+ 13969us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:49.172227 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:49.172335 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:49.176344 (+ 4009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:49.176349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:49.176376 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:42:49.176663 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.176673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.178797 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.178802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.180980 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.180995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.183595 (+ 2600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.183603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.183869 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.183947 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 23:42:49.184041 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 23:42:49.184229 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13854,"thread_start_us":13710,"threads_started":1}
I20260808 23:42:49.184839 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.132710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51792 (local address 127.18.186.131:44915)
0808 23:42:49.168006 (+ 35296us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.168019 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.168068 (+ 49us) server_negotiation.cc:407] Connection header received
0808 23:42:49.168218 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.168231 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.168592 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.170539 (+ 1947us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:42:49.170943 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.176875 (+ 5932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.180154 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.181794 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.183988 (+ 2194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.184176 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.184257 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.184440 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35075,"thread_start_us":34912,"threads_started":1}
I20260808 23:42:49.186100 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.158411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.18.186.130:34933)
0808 23:42:49.174958 (+ 16547us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.174967 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.174999 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:42:49.175104 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.175114 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.175383 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.175628 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:42:49.177072 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.178634 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.181213 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.183243 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.185260 (+ 2017us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.185495 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.185587 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.185775 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16339,"thread_start_us":18288,"threads_started":1}
I20260808 23:42:49.186478 904 tablet_service.cc:1467] Processing CreateTablet for tablet a6fe7c8e95a14d4b951f91a70df0353d (DEFAULT_TABLE table=test-table [id=36f36f6fe11e4fadaaa0418da74157e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:42:49.189219 904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6fe7c8e95a14d4b951f91a70df0353d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:49.188763 1208 tablet_service.cc:1467] Processing CreateTablet for tablet a6fe7c8e95a14d4b951f91a70df0353d (DEFAULT_TABLE table=test-table [id=36f36f6fe11e4fadaaa0418da74157e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:42:49.190158 1208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6fe7c8e95a14d4b951f91a70df0353d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:49.190023 1067 tablet_service.cc:1467] Processing CreateTablet for tablet a6fe7c8e95a14d4b951f91a70df0353d (DEFAULT_TABLE table=test-table [id=36f36f6fe11e4fadaaa0418da74157e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:42:49.191557 1067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6fe7c8e95a14d4b951f91a70df0353d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:49.213069 1312 tablet_bootstrap.cc:492] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Bootstrap starting.
I20260808 23:42:49.213073 1310 tablet_bootstrap.cc:492] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Bootstrap starting.
I20260808 23:42:49.217519 1311 tablet_bootstrap.cc:492] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Bootstrap starting.
I20260808 23:42:49.218292 1312 tablet_bootstrap.cc:654] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:49.218592 1310 tablet_bootstrap.cc:654] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:49.219924 1312 log.cc:824] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:49.221251 1311 tablet_bootstrap.cc:654] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:49.221676 1310 log.cc:824] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:49.222460 1311 log.cc:824] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:49.229291 1310 tablet_bootstrap.cc:492] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: No bootstrap required, opened a new log
I20260808 23:42:49.229770 1310 ts_tablet_manager.cc:1397] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260808 23:42:49.234338 1312 tablet_bootstrap.cc:492] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: No bootstrap required, opened a new log
I20260808 23:42:49.234719 1312 ts_tablet_manager.cc:1397] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.008s
I20260808 23:42:49.238963 1311 tablet_bootstrap.cc:492] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: No bootstrap required, opened a new log
I20260808 23:42:49.239257 1311 ts_tablet_manager.cc:1397] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.008s
I20260808 23:42:49.244166 1310 raft_consensus.cc:348] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.244786 1310 raft_consensus.cc:374] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:49.244946 1310 raft_consensus.cc:729] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6df6b2f44f4d95afbc0f9f0e2fabc0, State: Initialized, Role: FOLLOWER
I20260808 23:42:49.245808 1310 consensus_queue.cc:260] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.247987 1312 raft_consensus.cc:348] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.248627 1312 raft_consensus.cc:374] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:49.248822 1312 raft_consensus.cc:729] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39288e833d934b91aae1535e718dd3fe, State: Initialized, Role: FOLLOWER
I20260808 23:42:49.249060 1311 raft_consensus.cc:348] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.249493 1311 raft_consensus.cc:374] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:49.249617 1311 raft_consensus.cc:729] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944fd755d8f1403d8c2ee55229f69295, State: Initialized, Role: FOLLOWER
I20260808 23:42:49.249876 1312 consensus_queue.cc:260] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.250276 1311 consensus_queue.cc:260] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.254487 1310 ts_tablet_manager.cc:1428] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Time spent starting tablet: real 0.025s user 0.014s sys 0.006s
I20260808 23:42:49.254977 1281 heartbeater.cc:503] Master 127.18.186.190:37209 was elected leader, sending a full tablet report...
I20260808 23:42:49.255256 1312 ts_tablet_manager.cc:1428] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Time spent starting tablet: real 0.020s user 0.011s sys 0.007s
I20260808 23:42:49.255609 980 heartbeater.cc:503] Master 127.18.186.190:37209 was elected leader, sending a full tablet report...
I20260808 23:42:49.260012 1311 ts_tablet_manager.cc:1428] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Time spent starting tablet: real 0.021s user 0.012s sys 0.007s
W20260808 23:42:49.341805 1140 tablet.cc:2369] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:42:49.395833 1318 raft_consensus.cc:482] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:42:49.396224 1318 raft_consensus.cc:504] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.398156 1318 leader_election.cc:288] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a6df6b2f44f4d95afbc0f9f0e2fabc0 (127.18.186.131:44915), 39288e833d934b91aae1535e718dd3fe (127.18.186.129:32933)
I20260808 23:42:49.407627 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.398824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:35423 (local address 127.18.186.131:44915)
0808 23:42:49.399130 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.399140 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.399918 (+ 778us) server_negotiation.cc:407] Connection header received
0808 23:42:49.400239 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.400249 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.400322 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.400430 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:42:49.401179 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.402411 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.405026 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.406250 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.406378 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.406453 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.407150 (+ 697us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.407310 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 23:42:49.407970 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.398675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:32933 (local address 127.18.186.130:52095)
0808 23:42:49.400126 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:49.400144 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:49.400331 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:49.400858 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:49.400862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:49.400887 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:42:49.401283 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.401294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.402767 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.402773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.404519 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.404532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.406803 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.406815 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.406991 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.407020 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 23:42:49.407199 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 23:42:49.407384 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1334,"mutex_wait_us":21,"thread_start_us":775,"threads_started":1}
I20260808 23:42:49.407996 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.399193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:52095 (local address 127.18.186.129:32933)
0808 23:42:49.399393 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.399399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.400227 (+ 828us) server_negotiation.cc:407] Connection header received
0808 23:42:49.400494 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.400500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.400573 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.400631 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:42:49.401477 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.402623 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.404680 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.406391 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.406585 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.406683 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.407559 (+ 876us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.407697 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:42:49.408488 1320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.398625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:44915 (local address 127.18.186.130:35423)
0808 23:42:49.399779 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:49.399811 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:49.400060 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:49.400446 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:49.400451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:49.400477 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:42:49.401015 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.401027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.402560 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.402566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.404831 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:49.404846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.406676 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.406683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:49.406927 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.406995 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 23:42:49.407097 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 23:42:49.407231 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":3,"thread_start_us":1188,"threads_started":1}
I20260808 23:42:49.409278 927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6fe7c8e95a14d4b951f91a70df0353d" candidate_uuid: "944fd755d8f1403d8c2ee55229f69295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39288e833d934b91aae1535e718dd3fe" is_pre_election: true
I20260808 23:42:49.409686 1231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6fe7c8e95a14d4b951f91a70df0353d" candidate_uuid: "944fd755d8f1403d8c2ee55229f69295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" is_pre_election: true
I20260808 23:42:49.410008 927 raft_consensus.cc:2454] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 944fd755d8f1403d8c2ee55229f69295 in term 0.
I20260808 23:42:49.410291 1231 raft_consensus.cc:2454] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 944fd755d8f1403d8c2ee55229f69295 in term 0.
I20260808 23:42:49.410807 1014 leader_election.cc:302] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39288e833d934b91aae1535e718dd3fe, 944fd755d8f1403d8c2ee55229f69295; no voters:
I20260808 23:42:49.411275 1318 raft_consensus.cc:2781] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:42:49.411454 1318 raft_consensus.cc:482] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:42:49.411571 1318 raft_consensus.cc:3037] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:49.414002 1318 raft_consensus.cc:504] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.414678 1318 leader_election.cc:288] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [CANDIDATE]: Term 1 election: Requested vote from peers 5a6df6b2f44f4d95afbc0f9f0e2fabc0 (127.18.186.131:44915), 39288e833d934b91aae1535e718dd3fe (127.18.186.129:32933)
I20260808 23:42:49.415088 1231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6fe7c8e95a14d4b951f91a70df0353d" candidate_uuid: "944fd755d8f1403d8c2ee55229f69295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0"
I20260808 23:42:49.415195 927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6fe7c8e95a14d4b951f91a70df0353d" candidate_uuid: "944fd755d8f1403d8c2ee55229f69295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39288e833d934b91aae1535e718dd3fe"
I20260808 23:42:49.415333 1231 raft_consensus.cc:3037] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:49.415405 927 raft_consensus.cc:3037] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:49.419056 1231 raft_consensus.cc:2454] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 944fd755d8f1403d8c2ee55229f69295 in term 1.
I20260808 23:42:49.419247 927 raft_consensus.cc:2454] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 944fd755d8f1403d8c2ee55229f69295 in term 1.
I20260808 23:42:49.419628 1016 leader_election.cc:302] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a6df6b2f44f4d95afbc0f9f0e2fabc0, 944fd755d8f1403d8c2ee55229f69295; no voters:
I20260808 23:42:49.419945 1318 raft_consensus.cc:2781] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:42:49.420514 1318 raft_consensus.cc:686] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 1 LEADER]: Becoming Leader. State: Replica: 944fd755d8f1403d8c2ee55229f69295, State: Running, Role: LEADER
I20260808 23:42:49.421277 1318 consensus_queue.cc:237] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } }
I20260808 23:42:49.426131 759 catalog_manager.cc:5495] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 reported cstate change: term changed from 0 to 1, leader changed from <none> to 944fd755d8f1403d8c2ee55229f69295 (127.18.186.130). New cstate: current_term: 1 leader_uuid: "944fd755d8f1403d8c2ee55229f69295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "944fd755d8f1403d8c2ee55229f69295" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 } health_report { overall_health: UNKNOWN } } }
W20260808 23:42:49.454691 981 tablet.cc:2369] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:42:49.461506 1283 tablet.cc:2369] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:42:49.465623 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.459534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33066 (local address 127.18.186.130:34933)
0808 23:42:49.459811 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.459819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.459841 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:42:49.460076 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.460080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.460188 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.460363 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:42:49.460823 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.461979 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.463375 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.463889 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.463993 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.464429 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:42:49.465000 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:42:49.465122 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.465226 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.465306 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:42:49.473831 927 raft_consensus.cc:1261] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 1 FOLLOWER]: Refusing update from remote peer 944fd755d8f1403d8c2ee55229f69295: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:42:49.473835 1231 raft_consensus.cc:1261] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 1 FOLLOWER]: Refusing update from remote peer 944fd755d8f1403d8c2ee55229f69295: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:42:49.474875 1318 consensus_queue.cc:1035] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39288e833d934b91aae1535e718dd3fe" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:42:49.475155 1321 consensus_queue.cc:1035] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a6df6b2f44f4d95afbc0f9f0e2fabc0" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 44915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:42:49.486735 1326 mvcc.cc:205] Tried to move back new op lower bound from 7316408604553867264 to 7316408604353314816. Current Snapshot: MvccSnapshot[applied={T|T < 7316408604553867264}]
I20260808 23:42:49.497305 1327 mvcc.cc:205] Tried to move back new op lower bound from 7316408604553867264 to 7316408604353314816. Current Snapshot: MvccSnapshot[applied={T|T < 7316408604553867264}]
I20260808 23:42:49.503568 759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:42:49.503826 759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:42:49.507241 759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65d4dad255e348bf869b769fa27bc2e4: Sending DeleteTablet for 3 replicas of tablet a6fe7c8e95a14d4b951f91a70df0353d
I20260808 23:42:49.508491 1208 tablet_service.cc:1514] Processing DeleteTablet for tablet a6fe7c8e95a14d4b951f91a70df0353d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:42:49 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:51792
I20260808 23:42:49.508639 1067 tablet_service.cc:1514] Processing DeleteTablet for tablet a6fe7c8e95a14d4b951f91a70df0353d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:42:49 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:33052
I20260808 23:42:49.509248 904 tablet_service.cc:1514] Processing DeleteTablet for tablet a6fe7c8e95a14d4b951f91a70df0353d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:42:49 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:36498
I20260808 23:42:49.509402 1336 tablet_replica.cc:330] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: stopping tablet replica
I20260808 23:42:49.509825 1336 raft_consensus.cc:2229] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:42:49.510299 1336 raft_consensus.cc:2258] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:42:49.512416 1336 ts_tablet_manager.cc:1905] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:42:49.512593 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 800
I20260808 23:42:49.522481 1336 ts_tablet_manager.cc:1918] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:42:49.522693 1336 log.cc:1196] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/a6fe7c8e95a14d4b951f91a70df0353d
I20260808 23:42:49.523373 1336 ts_tablet_manager.cc:1939] T a6fe7c8e95a14d4b951f91a70df0353d P 5a6df6b2f44f4d95afbc0f9f0e2fabc0: Deleting consensus metadata
I20260808 23:42:49.525673 746 catalog_manager.cc:4841] TS 5a6df6b2f44f4d95afbc0f9f0e2fabc0 (127.18.186.131:44915): tablet a6fe7c8e95a14d4b951f91a70df0353d (table test-table [id=36f36f6fe11e4fadaaa0418da74157e7]) successfully deleted
I20260808 23:42:49.531703 1338 tablet_replica.cc:330] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: stopping tablet replica
I20260808 23:42:49.532107 1338 raft_consensus.cc:2229] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:42:49.532558 1338 raft_consensus.cc:2258] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:42:49.533366 1337 tablet_replica.cc:330] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: stopping tablet replica
I20260808 23:42:49.533783 1337 raft_consensus.cc:2229] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:42:49.534487 1337 raft_consensus.cc:2258] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:42:49.535099 1338 ts_tablet_manager.cc:1905] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:42:49.536700 1337 ts_tablet_manager.cc:1905] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:42:49.542445 1338 ts_tablet_manager.cc:1918] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:42:49.542619 1338 log.cc:1196] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/a6fe7c8e95a14d4b951f91a70df0353d
I20260808 23:42:49.543262 1338 ts_tablet_manager.cc:1939] T a6fe7c8e95a14d4b951f91a70df0353d P 39288e833d934b91aae1535e718dd3fe: Deleting consensus metadata
I20260808 23:42:49.544380 1337 ts_tablet_manager.cc:1918] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:42:49.544549 1337 log.cc:1196] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/a6fe7c8e95a14d4b951f91a70df0353d
I20260808 23:42:49.545099 1337 ts_tablet_manager.cc:1939] T a6fe7c8e95a14d4b951f91a70df0353d P 944fd755d8f1403d8c2ee55229f69295: Deleting consensus metadata
I20260808 23:42:49.547065 744 catalog_manager.cc:4841] TS 39288e833d934b91aae1535e718dd3fe (127.18.186.129:32933): tablet a6fe7c8e95a14d4b951f91a70df0353d (table test-table [id=36f36f6fe11e4fadaaa0418da74157e7]) successfully deleted
I20260808 23:42:49.548870 746 catalog_manager.cc:4841] TS 944fd755d8f1403d8c2ee55229f69295 (127.18.186.130:34933): tablet a6fe7c8e95a14d4b951f91a70df0353d (table test-table [id=36f36f6fe11e4fadaaa0418da74157e7]) successfully deleted
I20260808 23:42:49.550552 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.513594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.18.186.129:32933)
0808 23:42:49.513794 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.513800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.528132 (+ 14332us) server_negotiation.cc:407] Connection header received
0808 23:42:49.528493 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.528503 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.528617 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.528722 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:49.534913 (+ 6191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.536067 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.538098 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.538751 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.538920 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.542751 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:49.542767 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:49.542792 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:49.542844 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:49.546718 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:49.547590 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:49.547599 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:49.547602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:49.547766 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:49.549123 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:49.549128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:49.549130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:49.549485 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:49.549735 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.550071 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.550237 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:42:49.551283 975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:49.644320 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 800
I20260808 23:42:49.664145 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 984
I20260808 23:42:49.679060 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.664930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33080 (local address 127.18.186.130:34933)
0808 23:42:49.665130 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.665136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.665290 (+ 154us) server_negotiation.cc:407] Connection header received
0808 23:42:49.665500 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.665504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.665603 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.665720 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:49.667075 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.668088 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.669368 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.669813 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.669943 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.672602 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:49.672615 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:49.672633 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:49.672672 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:49.676146 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:49.676911 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:49.676921 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:49.676928 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:49.677078 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:49.677475 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:49.677482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:49.677485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:49.677856 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:49.678147 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.678453 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.678689 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:42:49.680136 1134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:49.794559 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 984
I20260808 23:42:49.815652 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 1143
I20260808 23:42:49.828955 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:49.816478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.18.186.131:44915)
0808 23:42:49.816690 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:49.816695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:49.816715 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:42:49.816809 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:49.816813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:49.816875 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:49.816955 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:49.818266 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.819008 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.820426 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:49.820802 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:49.820899 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:49.823395 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:49.823407 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:49.823464 (+ 57us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:49.823518 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:49.826115 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:49.826746 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:49.826753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:49.826755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:49.826919 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:49.827251 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:49.827254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:49.827256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:49.827942 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:49.828160 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:49.828415 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 23:42:49.828574 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 23:42:49.829625 1275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:49.918246 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 1143
I20260808 23:42:49.936949 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 724
I20260808 23:42:49.938149 792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:50.049502 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 724
I20260808 23:42:50.073781 19178 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 23:42:50.418833 19178 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 23:42:50.419086 19178 mini_ranger.cc:66] Stopping Ranger...
I20260808 23:42:50.764285 19178 mini_ranger.cc:68] Stopped Ranger
2026-08-08 23:42:50.764 UTC [32135] LOG: received smart shutdown request
2026-08-08 23:42:50.767 UTC [32135] LOG: background worker "logical replication launcher" (PID 32144) exited with exit code 1
2026-08-08 23:42:50.767 UTC [32139] LOG: shutting down
2026-08-08 23:42:50.925 UTC [32135] LOG: database system is shut down
2026-08-08T23:42:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52663 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:42:51 dist-test-slave-5k54 krb5kdc[1353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:42:51 dist-test-slave-5k54 krb5kdc[1353](info): set up 2 sockets
Aug 08 23:42:51 dist-test-slave-5k54 krb5kdc[1353](info): commencing operation
krb5kdc: starting...
W20260808 23:42:52.198922 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:42:52 dist-test-slave-5k54 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:42:52.454845 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:37841
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:37841,127.18.186.189:46161,127.18.186.188:37337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:42:52.784829 1368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:52.785109 1368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:52.785197 1368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:52.792140 1368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:42:52.792234 1368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:52.792279 1368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:52.792332 1368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:42:52.792375 1368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:42:52.801601 1368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:37841,127.18.186.189:46161,127.18.186.188:37337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:37841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:52.802793 1368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:52.804044 1368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:52.815670 1374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:52.815748 1373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:52.816182 1376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:52.816991 1368 server_base.cc:1034] running on GCE node
I20260808 23:42:52.817816 1368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:52.819200 1368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:52.820432 1368 hybrid_clock.cc:648] HybridClock initialized: now 1786232572820400 us; error 42 us; skew 500 ppm
I20260808 23:42:52.820779 1368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:52 dist-test-slave-5k54 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232572, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:52.827286 1368 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:42:52.828855 1368 webserver.cc:460] Webserver started at http://127.18.186.190:46641/ using document root <none> and password file <none>
I20260808 23:42:52.829576 1368 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:52.829716 1368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:52.830029 1368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:42:52.833094 1368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "4149f10c09b54eaca32dc2fcf6798225"
format_stamp: "Formatted at 2026-08-08 23:42:52 on dist-test-slave-5k54"
server_key: "a9a2aa293f55e736457e6fb35681b788"
server_key_iv: "9c6ecc8e78d2fb132f54ff37a13574e0"
server_key_version: "encryptionkey@0"
I20260808 23:42:52.833930 1368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "4149f10c09b54eaca32dc2fcf6798225"
format_stamp: "Formatted at 2026-08-08 23:42:52 on dist-test-slave-5k54"
server_key: "a9a2aa293f55e736457e6fb35681b788"
server_key_iv: "9c6ecc8e78d2fb132f54ff37a13574e0"
server_key_version: "encryptionkey@0"
I20260808 23:42:52.839200 1368 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 23:42:52.842800 1383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:52.844206 1368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:42:52.844393 1368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "4149f10c09b54eaca32dc2fcf6798225"
format_stamp: "Formatted at 2026-08-08 23:42:52 on dist-test-slave-5k54"
server_key: "a9a2aa293f55e736457e6fb35681b788"
server_key_iv: "9c6ecc8e78d2fb132f54ff37a13574e0"
server_key_version: "encryptionkey@0"
I20260808 23:42:52.844596 1368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:52.855695 1368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:52.859663 1368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:52.859964 1368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:52.884583 1368 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:37841
I20260808 23:42:52.884621 1441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:37841 every 8 connection(s)
I20260808 23:42:52.886014 1368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:42:52.887923 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 1368
I20260808 23:42:52.888172 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:42:52.889103 19178 external_mini_cluster.cc:1428] Setting key 83888003157fcd1c6f5445997cab9da2
I20260808 23:42:52.890689 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:52.901026 1442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.186.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.189@KRBTEST.COM" created.
Aug 08 23:42:52 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232572, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:52.928512 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:52.904610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.18.186.190:37841)
0808 23:42:52.906467 (+ 1857us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:52.906478 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:52.906539 (+ 61us) server_negotiation.cc:407] Connection header received
0808 23:42:52.908182 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:52.908192 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:52.908521 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:52.908736 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:52.909902 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:52.911929 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:52.913576 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:52.914644 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:52.914851 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:52.921286 (+ 6435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:52.921294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:52.921305 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:52.921335 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:52.924204 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:52.925176 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:52.925182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:52.925185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:52.925328 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:52.925789 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:52.925793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:52.925795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:52.926275 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:52.926432 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:52.927090 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 23:42:52.927352 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":1630,"spinlock_wait_cycles":25984,"thread_start_us":1430,"threads_started":1}
I20260808 23:42:52.928541 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:52.904611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37841 (local address 127.0.0.1:41092)
0808 23:42:52.905677 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:52.905747 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:52.907808 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:52.908773 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:52.908781 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:52.909354 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:52.909687 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:52.909704 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:52.912104 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:52.912108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:52.913427 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:52.913439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:52.914919 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:52.914925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:52.915139 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:52.915941 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:52.915963 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:52.921042 (+ 5079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:52.924334 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:52.924357 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:52.924372 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:52.925030 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:52.925440 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:52.925448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:52.925454 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:52.925668 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:52.926429 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:52.926439 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:52.926695 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 23:42:52.926986 (+ 291us) client_negotiation.cc:240] Negotiation successful
0808 23:42:52.927561 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":120,"thread_start_us":414,"threads_started":1}
I20260808 23:42:52.931663 1442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:42:52.932770 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:52.932430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46732)
0808 23:42:52.932579 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:52.932676 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 23:42:52.933048 1385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
I20260808 23:42:52.934548 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:52.934233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46740)
0808 23:42:52.934379 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:52.934438 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 23:42:52.934727 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46161: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab.
Entry for principal kudu/127.18.186.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab.
I20260808 23:42:52.951004 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.189:46161
--webserver_interface=127.18.186.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab
--principal=kudu/127.18.186.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:37841,127.18.186.189:46161,127.18.186.188:37337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:42:52.984115 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } attempt: 1
I20260808 23:42:52.985306 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:52.984833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46756)
0808 23:42:52.985036 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:52.985165 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 23:42:52.986233 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:52.985908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46768)
0808 23:42:52.986053 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:52.986116 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 23:42:52.986413 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46161: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
I20260808 23:42:53.054656 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } attempt: 2
I20260808 23:42:53.055872 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.055357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46774)
0808 23:42:53.055633 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.055724 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 23:42:53.056833 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.056481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46776)
0808 23:42:53.056631 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.056713 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 23:42:53.057020 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46161: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
I20260808 23:42:53.148315 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } attempt: 3
I20260808 23:42:53.149622 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.148997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46788)
0808 23:42:53.149285 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.149432 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 23:42:53.150764 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.150348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46804)
0808 23:42:53.150500 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.150595 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:42:53.151002 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46161: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
W20260808 23:42:53.286302 1449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:53.286576 1449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:53.286754 1449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:53.293820 1449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:42:53.293921 1449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:53.293968 1449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:53.294080 1449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:42:53.294159 1449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:42:53.294279 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } attempt: 4
I20260808 23:42:53.295640 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.294929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46816)
0808 23:42:53.295344 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.295478 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205}
I20260808 23:42:53.296716 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.296291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46828)
0808 23:42:53.296448 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.296563 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 23:42:53.296913 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46161: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46161: connect: Connection refused (error 111)
I20260808 23:42:53.305171 1449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:37841,127.18.186.189:46161,127.18.186.188:37337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab
--principal=kudu/127.18.186.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.189:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.186.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:53.306414 1449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:53.307730 1449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:53.319178 1454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:53.319147 1457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:53.319224 1455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:53.320147 1449 server_base.cc:1034] running on GCE node
I20260808 23:42:53.320930 1449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:53.322239 1449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:53.323522 1449 hybrid_clock.cc:648] HybridClock initialized: now 1786232573323474 us; error 32 us; skew 500 ppm
I20260808 23:42:53.323865 1449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:53.327056 1449 init.cc:377] Logged in from keytab as kudu/127.18.186.189@KRBTEST.COM (short username kudu)
I20260808 23:42:53.328476 1449 webserver.cc:460] Webserver started at http://127.18.186.189:38855/ using document root <none> and password file <none>
I20260808 23:42:53.329137 1449 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:53.329264 1449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:53.329571 1449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:42:53.332651 1449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/data/instance:
uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef"
format_stamp: "Formatted at 2026-08-08 23:42:53 on dist-test-slave-5k54"
server_key: "ce780e8e8a81b563c948c2837837dbca"
server_key_iv: "3f1713842bce9358b7971da58aa2094d"
server_key_version: "encryptionkey@0"
I20260808 23:42:53.333444 1449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/wal/instance:
uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef"
format_stamp: "Formatted at 2026-08-08 23:42:53 on dist-test-slave-5k54"
server_key: "ce780e8e8a81b563c948c2837837dbca"
server_key_iv: "3f1713842bce9358b7971da58aa2094d"
server_key_version: "encryptionkey@0"
I20260808 23:42:53.338354 1449 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:42:53.341699 1465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:53.343072 1449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:42:53.343240 1449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/wal
uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef"
format_stamp: "Formatted at 2026-08-08 23:42:53 on dist-test-slave-5k54"
server_key: "ce780e8e8a81b563c948c2837837dbca"
server_key_iv: "3f1713842bce9358b7971da58aa2094d"
server_key_version: "encryptionkey@0"
I20260808 23:42:53.343458 1449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:53.358232 1449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:53.362377 1449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:53.362663 1449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:53.380496 1449 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.189:46161
I20260808 23:42:53.380529 1516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.189:46161 every 8 connection(s)
I20260808 23:42:53.381825 1449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
I20260808 23:42:53.385521 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 1449
I20260808 23:42:53.385584 1517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:53.385737 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-1/wal/instance
I20260808 23:42:53.386312 19178 external_mini_cluster.cc:1428] Setting key e45224a4a0ab9f49e362e8a9521df1e0
I20260808 23:42:53.395710 1517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.186.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.188@KRBTEST.COM" created.
I20260808 23:42:53.417637 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.399168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.18.186.190:37841)
0808 23:42:53.399418 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.399447 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.400489 (+ 1042us) server_negotiation.cc:407] Connection header received
0808 23:42:53.402365 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.402374 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.402474 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.402591 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.404087 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.405275 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.406756 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.407610 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.407754 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.411608 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.411613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.411616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.411649 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.414445 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.415109 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.415112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.415114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.415256 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.415721 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.415725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.415727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.416073 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.416187 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.416916 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.417174 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 23:42:53.418177 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.399527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37841 (local address 127.0.0.1:41098)
0808 23:42:53.400306 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.400357 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:53.402101 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:53.402796 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:53.402804 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:53.403291 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:53.403915 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.403932 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.405429 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.405432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.406615 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.406627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.407802 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.407807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.407982 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.408640 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:53.408661 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:53.411363 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:53.414568 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.414574 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.414586 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.415004 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.415397 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.415400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.415402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.415621 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.416204 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:53.416215 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:53.416534 (+ 319us) client_negotiation.cc:769] Sending connection context
0808 23:42:53.416748 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 23:42:53.417097 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":341,"threads_started":1}
I20260808 23:42:53.419909 1517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:42:53.436257 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.420886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.18.186.189:46161)
0808 23:42:53.421624 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.421629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.421652 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:42:53.421692 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.421697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.421859 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.421974 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.422947 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.424052 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.425740 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.426698 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.426843 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.430358 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.430369 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.430381 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.430417 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.433253 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.434107 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.434114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.434116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.434225 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.434643 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.434646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.434648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.434973 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.435122 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.435615 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.435959 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":137,"server-negotiator.queue_time_us":592,"thread_start_us":423,"threads_started":1}
I20260808 23:42:53.436340 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.420727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46842)
0808 23:42:53.420903 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.420922 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:53.421037 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:53.422026 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:53.422034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:53.422553 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:53.422768 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.422782 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.424195 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.424202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.425611 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.425623 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.426831 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.426835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.426984 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.427600 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:53.427617 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:53.430157 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:53.433398 (+ 3241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.433406 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.433412 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.433992 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.434325 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.434332 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.434338 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.434539 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.435109 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:53.435117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:53.435306 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 23:42:53.435504 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 23:42:53.435846 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 23:42:53.438871 1517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:42:53.440135 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.439706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47178)
0808 23:42:53.439857 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.439991 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 23:42:53.440395 1469 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
I20260808 23:42:53.442013 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.441669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47184)
0808 23:42:53.441837 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.441916 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:42:53.442214 1517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab.
Entry for principal kudu/127.18.186.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab.
I20260808 23:42:53.450963 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.188:37337
--webserver_interface=127.18.186.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab
--principal=kudu/127.18.186.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:37841,127.18.186.189:46161,127.18.186.188:37337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:42:53.491531 1517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 1
I20260808 23:42:53.492641 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.492191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47200)
0808 23:42:53.492381 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.492495 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 23:42:53.493675 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.493345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47208)
0808 23:42:53.493510 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.493573 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 23:42:53.493932 1517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
I20260808 23:42:53.562206 1517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 2
I20260808 23:42:53.563413 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.562959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47212)
0808 23:42:53.563157 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.563254 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 23:42:53.564478 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.564136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47214)
0808 23:42:53.564290 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.564358 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:42:53.564741 1517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
I20260808 23:42:53.596165 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } attempt: 5
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232572, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:42:53.611102 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.597002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46846 (local address 127.18.186.189:46161)
0808 23:42:53.597207 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.597212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.597230 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:42:53.597440 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.597444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.597505 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.597585 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.598605 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.599297 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.601058 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.601482 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.601575 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.605611 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.605616 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.605619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.605651 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.608341 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.609085 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.609089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.609091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.609203 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.609643 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.609646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.609648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.609900 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.610015 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.610538 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.610705 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:42:53.611235 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.596866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46846)
0808 23:42:53.597113 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.597137 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:53.597298 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:53.597649 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:53.597654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:53.598162 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:53.598415 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.598437 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.599473 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.599478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.600923 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.600938 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.601629 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.601634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.601852 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.602478 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:53.602510 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:53.605421 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:53.608493 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.608498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.608502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.608980 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.609316 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.609332 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.609337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.609532 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.610034 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:53.610042 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:53.610252 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 23:42:53.610461 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 23:42:53.610834 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 23:42:53.612428 1442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:42:53.613504 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.613065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47230)
0808 23:42:53.613245 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.613351 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 23:42:53.614674 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.614174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47236)
0808 23:42:53.614394 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.614513 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 23:42:53.614969 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
I20260808 23:42:53.655969 1517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 3
I20260808 23:42:53.657244 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.656784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47252)
0808 23:42:53.656986 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.657088 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 23:42:53.658325 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.657885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47268)
0808 23:42:53.658139 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.658218 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260808 23:42:53.658561 1517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
I20260808 23:42:53.666181 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 1
I20260808 23:42:53.667335 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.666714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47272)
0808 23:42:53.666961 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.667105 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 23:42:53.668362 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.668006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47278)
0808 23:42:53.668150 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.668232 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 23:42:53.668573 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
I20260808 23:42:53.736810 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 2
I20260808 23:42:53.738116 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.737510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47282)
0808 23:42:53.737772 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.737904 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 23:42:53.739202 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.738788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47290)
0808 23:42:53.738942 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.739036 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 23:42:53.739475 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
W20260808 23:42:53.790298 1524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:53.790616 1524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:53.790705 1524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:53.797895 1524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:42:53.797995 1524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:53.798048 1524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:53.798105 1524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:42:53.798148 1524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:42:53.801759 1517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 4
I20260808 23:42:53.802908 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.802445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47302)
0808 23:42:53.802663 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.802752 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 23:42:53.803891 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.803589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47314)
0808 23:42:53.803733 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.803798 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 23:42:53.804129 1517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
I20260808 23:42:53.808205 1524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:37841,127.18.186.189:46161,127.18.186.188:37337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab
--principal=kudu/127.18.186.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.188:37337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.186.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:53.809453 1524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:53.810719 1524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:53.822469 1532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:53.822417 1530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:53.822635 1529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:53.823232 1524 server_base.cc:1034] running on GCE node
I20260808 23:42:53.824088 1524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:53.825613 1524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:53.826838 1524 hybrid_clock.cc:648] HybridClock initialized: now 1786232573826786 us; error 47 us; skew 500 ppm
I20260808 23:42:53.827190 1524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:53.830304 1524 init.cc:377] Logged in from keytab as kudu/127.18.186.188@KRBTEST.COM (short username kudu)
I20260808 23:42:53.831772 1524 webserver.cc:460] Webserver started at http://127.18.186.188:34989/ using document root <none> and password file <none>
I20260808 23:42:53.832449 1524 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:53.832585 1524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:53.832880 1524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:42:53.835951 1524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/data/instance:
uuid: "1fecc3023c42459b8d5f7a1cc1af1354"
format_stamp: "Formatted at 2026-08-08 23:42:53 on dist-test-slave-5k54"
server_key: "d9548f86100e04fa7824330f1fea8912"
server_key_iv: "fab364039ff2c1b70824f8bd444a1e71"
server_key_version: "encryptionkey@0"
I20260808 23:42:53.836746 1524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/wal/instance:
uuid: "1fecc3023c42459b8d5f7a1cc1af1354"
format_stamp: "Formatted at 2026-08-08 23:42:53 on dist-test-slave-5k54"
server_key: "d9548f86100e04fa7824330f1fea8912"
server_key_iv: "fab364039ff2c1b70824f8bd444a1e71"
server_key_version: "encryptionkey@0"
I20260808 23:42:53.841820 1524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:42:53.845249 1540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:53.845705 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 3
I20260808 23:42:53.846714 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.846303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47328)
0808 23:42:53.846484 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.846565 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 23:42:53.846783 1524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:42:53.847050 1524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/wal
uuid: "1fecc3023c42459b8d5f7a1cc1af1354"
format_stamp: "Formatted at 2026-08-08 23:42:53 on dist-test-slave-5k54"
server_key: "d9548f86100e04fa7824330f1fea8912"
server_key_iv: "fab364039ff2c1b70824f8bd444a1e71"
server_key_version: "encryptionkey@0"
I20260808 23:42:53.847292 1524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:53.847759 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.847375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47334)
0808 23:42:53.847600 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.847669 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260808 23:42:53.847991 1442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:37337: Network error: Client connection negotiation failed: client connection to 127.18.186.188:37337: connect: Connection refused (error 111)
I20260808 23:42:53.860842 1524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:53.864646 1524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:53.864923 1524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:53.890820 1524 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.188:37337
I20260808 23:42:53.890851 1599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.188:37337 every 8 connection(s)
I20260808 23:42:53.892732 1524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
I20260808 23:42:53.896708 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 1524
I20260808 23:42:53.896893 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786232366675349-19178-0/minicluster-data/master-2/wal/instance
I20260808 23:42:53.897119 1600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:53.897562 19178 external_mini_cluster.cc:1428] Setting key f37ea5ac3a242ed0520e192535c0a338
I20260808 23:42:53.908777 1600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:53.925830 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.902366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.18.186.188:37337)
0808 23:42:53.903276 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.903294 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.903344 (+ 50us) server_negotiation.cc:407] Connection header received
0808 23:42:53.905634 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.905675 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.906209 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.907592 (+ 1383us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.908201 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.910368 (+ 2167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.911931 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.912729 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.912927 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.918168 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.918175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.918186 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.918226 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.921555 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.922271 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.922280 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.922288 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.922445 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.922994 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.922998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.923000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.923698 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.923954 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.924402 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.924699 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":375,"threads_started":1}
I20260808 23:42:53.928903 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.911908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.18.186.190:37841)
0808 23:42:53.912105 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.912111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.912940 (+ 829us) server_negotiation.cc:407] Connection header received
0808 23:42:53.913220 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.913226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.913306 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.913411 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.914612 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.915313 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.916878 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.917414 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.917518 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.921491 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.921497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.921502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.921544 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.925068 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.925790 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.925795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.925802 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.925935 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.927315 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.927318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.927320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.927650 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.927787 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.928292 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.928521 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:42:53.928916 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.911884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:37841 (local address 127.0.0.1:41110)
0808 23:42:53.912798 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.912847 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:53.913082 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:53.913469 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:53.913476 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:53.914172 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:53.914443 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.914452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.915506 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.915512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.916678 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.916702 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.917651 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.917659 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.917855 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.918527 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:53.918556 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:53.921284 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:53.925198 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.925211 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.925218 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.925659 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.927002 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.927010 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.927016 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.927184 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.927773 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:53.927780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:53.927993 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 23:42:53.928178 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 23:42:53.928500 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":496,"threads_started":1}
I20260808 23:42:53.930706 1600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:42:53.944965 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.931579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46161 (local address 127.0.0.1:46862)
0808 23:42:53.931814 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.931831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:53.931941 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:53.932231 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:53.932236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:53.932651 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:53.932845 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.932854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.934219 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.934224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.935534 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.935562 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.936516 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.936520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.936671 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.937202 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:53.937218 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:53.939572 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:53.942493 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.942497 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.942501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.942856 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.943295 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.943298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.943300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.943503 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.944000 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:53.944006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:53.944175 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 23:42:53.944323 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:42:53.944601 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260808 23:42:53.944984 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.928872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.18.186.190:37841)
0808 23:42:53.929311 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.929316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.929529 (+ 213us) server_negotiation.cc:407] Connection header received
0808 23:42:53.929697 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.929701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.929756 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.929838 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.931453 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.932665 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.934414 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.935024 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.935145 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.939524 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.939530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.939536 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.939571 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.942322 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.942952 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.942960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.942966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.943087 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.943508 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.943512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.943515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.943810 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.944129 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.944393 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.944601 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":572,"threads_started":1}
I20260808 23:42:53.945508 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.931545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.18.186.189:46161)
0808 23:42:53.931704 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.931709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.931896 (+ 187us) server_negotiation.cc:407] Connection header received
0808 23:42:53.932047 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.932050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.932107 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.932180 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.933008 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.934060 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.935687 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.936365 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.936519 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.939746 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.939751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.939754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.939785 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.942338 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.943004 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.943008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.943010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.943179 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.943613 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.943617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.943619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.943857 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.944023 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.944460 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.944698 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:42:53.946475 1600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
Aug 08 23:42:53 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:42:53.960232 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.947195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47366)
0808 23:42:53.947371 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:53.947390 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:53.947521 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:53.947857 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:53.947865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:53.948299 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:53.948444 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.948452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.949586 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.949592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.950672 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:53.950683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.951626 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.951631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:53.951873 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.952488 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:53.952504 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:53.954594 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:53.957964 (+ 3370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.957971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.957975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.958305 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.958627 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:53.958634 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:53.958639 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:53.958760 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:53.959229 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:53.959234 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:53.959367 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 23:42:53.959547 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 23:42:53.959839 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:42:53.960305 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.947341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.18.186.188:37337)
0808 23:42:53.947558 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.947565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.947584 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:42:53.947635 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.947640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.947710 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.947804 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.948573 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.949463 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.950801 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.951487 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.951700 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.954781 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.954789 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.954796 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.954837 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.957831 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.958410 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.958417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.958423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.958530 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.958854 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.958862 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.958865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.959125 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.959264 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.959645 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.960000 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":130,"server-negotiator.queue_time_us":112}
I20260808 23:42:53.962675 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.946695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.18.186.189:46161)
0808 23:42:53.947058 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.947063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.947078 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:42:53.947123 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.947128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.947201 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.947290 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.948936 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.949816 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.951303 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.952143 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.952243 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.957304 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.957308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.957311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.957339 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.959918 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.960687 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.960692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.960695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.960856 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:53.961268 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:53.961271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:53.961273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:53.961617 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:53.961886 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:53.962144 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 23:42:53.962365 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260808 23:42:53.965503 1600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: Bootstrap starting.
I20260808 23:42:53.967219 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:42:53.968997 1600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:53.970153 1600 log.cc:824] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:53.973480 1600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: No bootstrap required, opened a new log
I20260808 23:42:53.988051 1600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:53.988615 1600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:53.988795 1600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fecc3023c42459b8d5f7a1cc1af1354, State: Initialized, Role: FOLLOWER
I20260808 23:42:53.989596 1600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:53.996922 1616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:53.997391 1616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:42:53.999392 1600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:42:54.002650 1446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.972477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.18.186.190:37841)
0808 23:42:53.974494 (+ 2017us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.974501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.983196 (+ 8695us) server_negotiation.cc:407] Connection header received
0808 23:42:53.983406 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.983415 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.983551 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.983679 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.986509 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.987709 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.990064 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.990708 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.990804 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.995242 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.995248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.995253 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.995294 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:53.999199 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.000169 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.000176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.000182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.000364 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.000837 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.000842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.000845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.001218 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:54.001454 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:54.002037 (+ 583us) server_negotiation.cc:299] Negotiation successful
0808 23:42:54.002290 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1871}
I20260808 23:42:54.003417 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.973036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46876 (local address 127.18.186.189:46161)
0808 23:42:53.977446 (+ 4410us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.977454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.984002 (+ 6548us) server_negotiation.cc:407] Connection header received
0808 23:42:53.984275 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.984282 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.984364 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.984471 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.986776 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.987837 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.990064 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.990711 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.990804 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.995241 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.995248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.995253 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.995294 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:54.000149 (+ 4855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.001011 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.001017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.001019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.001165 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.001681 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.001686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.001689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.002066 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:54.002270 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:54.002841 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 23:42:54.003068 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4235}
I20260808 23:42:54.005012 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:53.972610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.18.186.188:37337)
0808 23:42:53.972869 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:53.972878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:53.984405 (+ 11527us) server_negotiation.cc:407] Connection header received
0808 23:42:53.984659 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:53.984664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:53.984739 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:53.984814 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:53.987080 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.988135 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.990425 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:53.991436 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:53.991559 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:53.996838 (+ 5279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:53.996843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:53.996847 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:53.996887 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:54.000551 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.001695 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.001704 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.001710 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.001917 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.002504 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.002512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.002518 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.002996 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:54.003284 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:54.004180 (+ 896us) server_negotiation.cc:299] Negotiation successful
0808 23:42:54.004482 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 23:42:54.016445 1627 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:42:54.016673 1627 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:42:54.012863 1524 master_runner.cc:186] Error getting master registration for 127.18.186.190:37841: OK, instance_id { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" instance_seqno: 1786232572878353 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:42:54.021426 1524 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:42:54.025215 1442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 4
Aug 08 23:42:54 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232572, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
I20260808 23:42:54.038957 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:54.025830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.18.186.188:37337)
0808 23:42:54.026036 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:54.026042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:54.026060 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:42:54.026231 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:54.026236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:54.026309 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:54.026407 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:54.027180 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:54.028023 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:54.029223 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:54.029674 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:54.029804 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:54.033514 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:54.033520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:54.033525 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:54.033559 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:54.036362 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.036974 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.036982 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.036985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.037121 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.037493 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.037500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.037503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.037797 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:54.037960 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:54.038394 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 23:42:54.038598 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 23:42:54.038980 1445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:54.025778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47398)
0808 23:42:54.025976 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:54.025997 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:54.026112 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:54.026452 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:54.026458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:54.026875 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:54.027044 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:54.027053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:54.028151 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:54.028157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:54.029082 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:54.029092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:54.029825 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:54.029831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:54.030023 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:54.030633 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:54.030650 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:54.033332 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:54.036486 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:54.036494 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:54.036499 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:54.036864 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:54.037235 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:54.037241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:54.037245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:54.037390 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:54.037949 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:54.037954 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:54.038116 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 23:42:54.038290 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 23:42:54.038575 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 23:42:54.044929 1442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: Bootstrap starting.
I20260808 23:42:54.049037 1442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:54.050390 1442 log.cc:824] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:54.058308 1442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: No bootstrap required, opened a new log
I20260808 23:42:54.069412 1442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:54.069801 1442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:54.069983 1442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4149f10c09b54eaca32dc2fcf6798225, State: Initialized, Role: FOLLOWER
I20260808 23:42:54.070650 1442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:54.071580 1630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.071905 1630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:42:54.072966 1442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 23:42:54.086328 1641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:42:54.086542 1641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:42:54.082454 1368 master_runner.cc:186] Error getting master registration for 127.18.186.189:46161: OK, instance_id { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" instance_seqno: 1786232573375627 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:42:54.092989 1368 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:42:54.103346 1517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } attempt: 5
Aug 08 23:42:54 dist-test-slave-5k54 krb5kdc[1353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
W20260808 23:42:54.112655 1390 tablet.cc:2369] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:42:54.116895 1547 tablet.cc:2369] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:42:54.118860 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:54.104050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47412 (local address 127.18.186.188:37337)
0808 23:42:54.104244 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:54.104250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:54.104346 (+ 96us) server_negotiation.cc:407] Connection header received
0808 23:42:54.104539 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:54.104544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:54.104615 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:54.104712 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:54.105586 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:54.106377 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:54.108187 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:54.108886 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:54.109002 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:54.112598 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:54.112608 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:54.112614 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:54.112665 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:54.115896 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.116554 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.116561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.116565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.116753 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:54.117249 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:54.117255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:54.117258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:54.117634 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:54.117782 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:54.118240 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 23:42:54.118477 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:42:54.118844 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:54.103997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:37337 (local address 127.0.0.1:47412)
0808 23:42:54.104247 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 23:42:54.104268 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:42:54.104420 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:42:54.104742 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:42:54.104747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:42:54.105284 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:42:54.105450 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:54.105459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:54.106570 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:54.106576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:54.107993 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:42:54.108010 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:42:54.109261 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:42:54.109267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:42:54.109527 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:54.110186 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:42:54.110204 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:42:54.112363 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:42:54.116027 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:54.116034 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:54.116039 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:54.116440 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:54.116924 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:42:54.116931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:42:54.116934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:42:54.117107 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:42:54.117844 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:42:54.117865 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:42:54.117996 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 23:42:54.118144 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:42:54.118445 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260808 23:42:54.125581 1517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef: Bootstrap starting.
I20260808 23:42:54.128728 1517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:54.129845 1517 log.cc:824] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:54.132544 1517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef: No bootstrap required, opened a new log
I20260808 23:42:54.141417 1517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:54.141767 1517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:54.141913 1517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dfc1b81eb3b4953a2bbf8ee34c4e7ef, State: Initialized, Role: FOLLOWER
I20260808 23:42:54.142509 1517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:54.143460 1644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.143898 1644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:42:54.144618 1517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:42:54.162667 1449 master_runner.cc:263] No leader master found from master 4dfc1b81eb3b4953a2bbf8ee34c4e7ef
I20260808 23:42:54.162880 1449 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 23:42:54.170387 1655 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:42:54.170629 1655 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:42:54.178906 1644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:42:54.179198 1644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:54.180977 1644 leader_election.cc:288] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4149f10c09b54eaca32dc2fcf6798225 (127.18.186.190:37841), 1fecc3023c42459b8d5f7a1cc1af1354 (127.18.186.188:37337)
I20260808 23:42:54.181632 1413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4149f10c09b54eaca32dc2fcf6798225" is_pre_election: true
I20260808 23:42:54.182179 1413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dfc1b81eb3b4953a2bbf8ee34c4e7ef in term 0.
I20260808 23:42:54.182785 1466 leader_election.cc:302] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4149f10c09b54eaca32dc2fcf6798225, 4dfc1b81eb3b4953a2bbf8ee34c4e7ef; no voters:
I20260808 23:42:54.182863 1570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" is_pre_election: true
I20260808 23:42:54.183487 1570 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dfc1b81eb3b4953a2bbf8ee34c4e7ef in term 0.
I20260808 23:42:54.183938 1644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:42:54.184082 1644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:42:54.184186 1644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:54.187822 1644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:54.188966 1644 leader_election.cc:288] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [CANDIDATE]: Term 1 election: Requested vote from peers 4149f10c09b54eaca32dc2fcf6798225 (127.18.186.190:37841), 1fecc3023c42459b8d5f7a1cc1af1354 (127.18.186.188:37337)
I20260808 23:42:54.188948 1413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4149f10c09b54eaca32dc2fcf6798225"
I20260808 23:42:54.189178 1413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:54.191548 1413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dfc1b81eb3b4953a2bbf8ee34c4e7ef in term 1.
I20260808 23:42:54.192159 1466 leader_election.cc:302] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4149f10c09b54eaca32dc2fcf6798225, 4dfc1b81eb3b4953a2bbf8ee34c4e7ef; no voters:
I20260808 23:42:54.192273 1570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fecc3023c42459b8d5f7a1cc1af1354"
I20260808 23:42:54.192453 1644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:42:54.192507 1570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:54.196839 1570 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dfc1b81eb3b4953a2bbf8ee34c4e7ef in term 1.
I20260808 23:42:54.204567 1644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [term 1 LEADER]: Becoming Leader. State: Replica: 4dfc1b81eb3b4953a2bbf8ee34c4e7ef, State: Running, Role: LEADER
I20260808 23:42:54.205228 1644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } }
I20260808 23:42:54.222280 1657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dfc1b81eb3b4953a2bbf8ee34c4e7ef. Latest consensus state: current_term: 1 leader_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.222745 1657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:54.240085 1659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:42:54.242370 1659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:42:54.264742 1570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [term 1 FOLLOWER]: Refusing update from remote peer 4dfc1b81eb3b4953a2bbf8ee34c4e7ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:42:54.266192 1644 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:42:54.267861 1413 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [term 1 FOLLOWER]: Refusing update from remote peer 4dfc1b81eb3b4953a2bbf8ee34c4e7ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:42:54.269124 1657 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:42:54.310361 1616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dfc1b81eb3b4953a2bbf8ee34c4e7ef. Latest consensus state: current_term: 1 leader_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.310698 1616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:42:54.315464 1662 mvcc.cc:205] Tried to move back new op lower bound from 7316408624177332224 to 7316408623950585856. Current Snapshot: MvccSnapshot[applied={T|T < 7316408624177332224}]
I20260808 23:42:54.323830 1657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.324221 1657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:54.328087 1659 catalog_manager.cc:1347] Generated new cluster ID: ced4c7f0ec044134ac726568eeb11cf9
I20260808 23:42:54.328198 1659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:42:54.331290 1630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dfc1b81eb3b4953a2bbf8ee34c4e7ef. Latest consensus state: current_term: 1 leader_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.331651 1630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:42:54.332619 1616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.332950 1616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:42:54.335151 1644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.336202 1644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:54.339537 1661 mvcc.cc:205] Tried to move back new op lower bound from 7316408624177332224 to 7316408623950585856. Current Snapshot: MvccSnapshot[applied={T|T < 7316408624177332224}]
I20260808 23:42:54.344524 1630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4149f10c09b54eaca32dc2fcf6798225" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 37841 } } peers { permanent_uuid: "4dfc1b81eb3b4953a2bbf8ee34c4e7ef" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46161 } } peers { permanent_uuid: "1fecc3023c42459b8d5f7a1cc1af1354" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 37337 } } }
I20260808 23:42:54.344798 1630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:42:54.360143 1659 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:42:54.362756 1659 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:42:54.377827 1659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dfc1b81eb3b4953a2bbf8ee34c4e7ef: Generated new TSK 0
I20260808 23:42:54.378909 1659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:42:55.018839 1627 catalog_manager.cc:1259] Loaded cluster ID: ced4c7f0ec044134ac726568eeb11cf9
I20260808 23:42:55.019042 1627 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: loading cluster ID for follower catalog manager: success
I20260808 23:42:55.021509 1627 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: acquiring CA information for follower catalog manager: success
I20260808 23:42:55.023232 1627 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1fecc3023c42459b8d5f7a1cc1af1354: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:42:55.088713 1641 catalog_manager.cc:1259] Loaded cluster ID: ced4c7f0ec044134ac726568eeb11cf9
I20260808 23:42:55.088826 1641 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: loading cluster ID for follower catalog manager: success
I20260808 23:42:55.091204 1641 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: acquiring CA information for follower catalog manager: success
I20260808 23:42:55.093088 1641 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4149f10c09b54eaca32dc2fcf6798225: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:42:55.534562 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 1368
I20260808 23:42:55.536041 1436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:55.648046 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 1368
I20260808 23:42:55.673810 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 1449
I20260808 23:42:55.675007 1512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:55.783306 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 1449
W20260808 23:42:55.807229 1543 connection.cc:535] server connection from 127.0.0.1:47412 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 23:42:55.807648 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 1524
I20260808 23:42:55.808889 1593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:55.920135 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 1524
2026-08-08T23:42:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4870 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:42:56 dist-test-slave-5k54 krb5kdc[1679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:42:56 dist-test-slave-5k54 krb5kdc[1679](info): set up 2 sockets
Aug 08 23:42:56 dist-test-slave-5k54 krb5kdc[1679](info): commencing operation
krb5kdc: starting...
W20260808 23:42:57.082859 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.006s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:42:57 dist-test-slave-5k54 krb5kdc[1679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:42:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:42:57.347168 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:43085
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:43085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:42:57.690902 1696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:57.691376 1696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:57.691561 1696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:57.698717 1696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:42:57.698870 1696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:57.698969 1696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:57.699080 1696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:42:57.699160 1696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:42:57.709004 1696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:43085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:43085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:57.710282 1696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:57.711575 1696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:57.723156 1702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:57.723256 1703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:57.723368 1705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:57.724370 1696 server_base.cc:1034] running on GCE node
I20260808 23:42:57.725072 1696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:57.726406 1696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:57.727620 1696 hybrid_clock.cc:648] HybridClock initialized: now 1786232577727577 us; error 53 us; skew 500 ppm
I20260808 23:42:57.728000 1696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:57 dist-test-slave-5k54 krb5kdc[1679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232577, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:57.731261 1696 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:42:57.732920 1696 webserver.cc:460] Webserver started at http://127.18.186.190:46321/ using document root <none> and password file <none>
I20260808 23:42:57.733610 1696 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:57.733764 1696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:57.734074 1696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:42:57.736958 1696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "ad5af37c5027408c821b7d5a5a7cc2f0"
format_stamp: "Formatted at 2026-08-08 23:42:57 on dist-test-slave-5k54"
server_key: "5c0fbda31c0d5a312454771fe696f87f"
server_key_iv: "cdf6f499324862210c4ee33d15e3060a"
server_key_version: "encryptionkey@0"
I20260808 23:42:57.737715 1696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "ad5af37c5027408c821b7d5a5a7cc2f0"
format_stamp: "Formatted at 2026-08-08 23:42:57 on dist-test-slave-5k54"
server_key: "5c0fbda31c0d5a312454771fe696f87f"
server_key_iv: "cdf6f499324862210c4ee33d15e3060a"
server_key_version: "encryptionkey@0"
I20260808 23:42:57.742641 1696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 23:42:57.746004 1713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:57.747252 1696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:42:57.747488 1696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "ad5af37c5027408c821b7d5a5a7cc2f0"
format_stamp: "Formatted at 2026-08-08 23:42:57 on dist-test-slave-5k54"
server_key: "5c0fbda31c0d5a312454771fe696f87f"
server_key_iv: "cdf6f499324862210c4ee33d15e3060a"
server_key_version: "encryptionkey@0"
I20260808 23:42:57.747718 1696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:57.768471 1696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:57.772300 1696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:57.772578 1696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:57.791756 1696 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:43085
I20260808 23:42:57.791779 1769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:43085 every 8 connection(s)
I20260808 23:42:57.793085 1696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:42:57.796751 1770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:57.802120 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 1696
I20260808 23:42:57.802346 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:42:57.803238 19178 external_mini_cluster.cc:1428] Setting key 762597893627701b0e7e5d35ccbcd255
I20260808 23:42:57.808279 1770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0: Bootstrap starting.
I20260808 23:42:57.811972 1770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:57.813210 1770 log.cc:824] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:57.816390 1770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0: No bootstrap required, opened a new log
Aug 08 23:42:57 dist-test-slave-5k54 krb5kdc[1679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:57.827298 1770 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5af37c5027408c821b7d5a5a7cc2f0" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43085 } }
I20260808 23:42:57.827804 1770 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:57.827917 1770 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5af37c5027408c821b7d5a5a7cc2f0, State: Initialized, Role: FOLLOWER
I20260808 23:42:57.828616 1770 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5af37c5027408c821b7d5a5a7cc2f0" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43085 } }
I20260808 23:42:57.828922 1770 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:42:57.829068 1770 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:42:57.829221 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:57.805452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58460 (local address 127.18.186.190:43085)
0808 23:42:57.806342 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:57.806353 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:57.806392 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:42:57.808158 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:57.808179 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:57.808618 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:57.808931 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:57.810782 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:57.812213 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:57.814103 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:57.814915 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:57.815060 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:57.820117 (+ 5057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:57.820128 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:57.820147 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:57.820202 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:57.824360 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:57.825072 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:57.825080 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:57.825089 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:57.825324 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:57.825843 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:57.825848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:57.825852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:57.826581 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:57.826846 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:57.827378 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 23:42:57.827857 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":471,"threads_started":1}
I20260808 23:42:57.829273 1770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:57.831743 1770 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5af37c5027408c821b7d5a5a7cc2f0" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43085 } }
I20260808 23:42:57.832317 1770 leader_election.cc:302] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad5af37c5027408c821b7d5a5a7cc2f0; no voters:
I20260808 23:42:57.833036 1770 leader_election.cc:288] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:42:57.833151 1775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:42:57.833218 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:42:57.835216 1770 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:42:57.840024 1775 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [term 1 LEADER]: Becoming Leader. State: Replica: ad5af37c5027408c821b7d5a5a7cc2f0, State: Running, Role: LEADER
I20260808 23:42:57.840891 1775 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5af37c5027408c821b7d5a5a7cc2f0" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43085 } }
I20260808 23:42:57.844038 1776 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad5af37c5027408c821b7d5a5a7cc2f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5af37c5027408c821b7d5a5a7cc2f0" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43085 } } }
I20260808 23:42:57.844380 1776 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:57.844978 1777 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad5af37c5027408c821b7d5a5a7cc2f0. Latest consensus state: current_term: 1 leader_uuid: "ad5af37c5027408c821b7d5a5a7cc2f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5af37c5027408c821b7d5a5a7cc2f0" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43085 } } }
I20260808 23:42:57.845289 1777 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0 [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:57.845316 1790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:42:57.850116 1790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:42:57.857986 1790 catalog_manager.cc:1347] Generated new cluster ID: af9d18edfc5b404cad9731001afff80c
I20260808 23:42:57.858094 1790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:42:57.903790 1790 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:42:57.905719 1790 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:42:57.915331 1790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad5af37c5027408c821b7d5a5a7cc2f0: Generated new TSK 0
I20260808 23:42:57.916358 1790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:42:57.947021 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 1696
I20260808 23:42:57.948252 1765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:42:58.046972 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 1696
2026-08-08T23:42:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2124 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:42:58 dist-test-slave-5k54 krb5kdc[1801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:42:58 dist-test-slave-5k54 krb5kdc[1801](info): set up 2 sockets
Aug 08 23:42:58 dist-test-slave-5k54 krb5kdc[1801](info): commencing operation
krb5kdc: starting...
W20260808 23:42:59.209731 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.006s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:42:59 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:42:59.474908 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:45339
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:45339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:42:59.810163 1815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:42:59.810520 1815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:42:59.810691 1815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:42:59.818138 1815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:42:59.818312 1815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:42:59.818418 1815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:42:59.818527 1815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:42:59.818619 1815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:42:59.828899 1815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:45339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:45339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:42:59.830395 1815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:42:59.831738 1815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:42:59.843514 1823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:59.843469 1821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:42:59.843834 1820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:42:59.844537 1815 server_base.cc:1034] running on GCE node
I20260808 23:42:59.845305 1815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:42:59.846689 1815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:42:59.847918 1815 hybrid_clock.cc:648] HybridClock initialized: now 1786232579847869 us; error 56 us; skew 500 ppm
I20260808 23:42:59.848259 1815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:42:59 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232579, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:42:59.851486 1815 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:42:59.852955 1815 webserver.cc:460] Webserver started at http://127.18.186.190:33637/ using document root <none> and password file <none>
I20260808 23:42:59.853645 1815 fs_manager.cc:359] Metadata directory not provided
I20260808 23:42:59.853797 1815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:42:59.854103 1815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:42:59.857112 1815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "cec7f2500dd147f8be1c52680e0bff45"
format_stamp: "Formatted at 2026-08-08 23:42:59 on dist-test-slave-5k54"
server_key: "2c7aa51a13e5e3714441f98eb6dd96b5"
server_key_iv: "1ffe36b418bed15ed21d4e09a3f9181f"
server_key_version: "encryptionkey@0"
I20260808 23:42:59.857869 1815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "cec7f2500dd147f8be1c52680e0bff45"
format_stamp: "Formatted at 2026-08-08 23:42:59 on dist-test-slave-5k54"
server_key: "2c7aa51a13e5e3714441f98eb6dd96b5"
server_key_iv: "1ffe36b418bed15ed21d4e09a3f9181f"
server_key_version: "encryptionkey@0"
I20260808 23:42:59.862927 1815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 23:42:59.866292 1831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:42:59.867621 1815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:42:59.867802 1815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "cec7f2500dd147f8be1c52680e0bff45"
format_stamp: "Formatted at 2026-08-08 23:42:59 on dist-test-slave-5k54"
server_key: "2c7aa51a13e5e3714441f98eb6dd96b5"
server_key_iv: "1ffe36b418bed15ed21d4e09a3f9181f"
server_key_version: "encryptionkey@0"
I20260808 23:42:59.867997 1815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:42:59.902071 1815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:42:59.906220 1815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:42:59.906533 1815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:42:59.925823 1815 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:45339
I20260808 23:42:59.925859 1889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:45339 every 8 connection(s)
I20260808 23:42:59.927196 1815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:42:59.930894 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 1815
I20260808 23:42:59.930972 1890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:42:59.931171 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:42:59.931968 19178 external_mini_cluster.cc:1428] Setting key 06508f3039cfc95b6e6bd3a49cf7bc9f
I20260808 23:42:59.944875 1890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Bootstrap starting.
Aug 08 23:42:59 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:42:59.948549 1890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Neither blocks nor log segments found. Creating new log.
I20260808 23:42:59.949618 1890 log.cc:824] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Log is configured to *not* fsync() on all Append() calls
I20260808 23:42:59.952642 1890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: No bootstrap required, opened a new log
I20260808 23:42:59.956148 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:42:59.934099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34054 (local address 127.18.186.190:45339)
0808 23:42:59.934926 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0808 23:42:59.934938 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:42:59.934979 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:42:59.936932 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:42:59.936955 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:42:59.937344 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:42:59.937685 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:42:59.939531 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:59.940797 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:59.942737 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:42:59.943367 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:42:59.943600 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:42:59.948513 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:42:59.948520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:42:59.948537 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:42:59.948575 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:42:59.952024 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:59.952729 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:59.952741 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:59.952769 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:59.952947 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:42:59.953353 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:42:59.953358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:42:59.953364 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:42:59.953881 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:42:59.954226 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0808 23:42:59.954656 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 23:42:59.955095 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":365,"threads_started":1}
I20260808 23:42:59.960548 1890 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:42:59.960933 1890 raft_consensus.cc:374] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:42:59.961050 1890 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec7f2500dd147f8be1c52680e0bff45, State: Initialized, Role: FOLLOWER
I20260808 23:42:59.961683 1890 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:42:59.961930 1890 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:42:59.962088 1890 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:42:59.962252 1890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:42:59.964656 1890 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:42:59.965201 1890 leader_election.cc:302] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec7f2500dd147f8be1c52680e0bff45; no voters:
I20260808 23:42:59.965807 1890 leader_election.cc:288] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:42:59.965924 1895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:42:59.966646 1895 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 1 LEADER]: Becoming Leader. State: Replica: cec7f2500dd147f8be1c52680e0bff45, State: Running, Role: LEADER
I20260808 23:42:59.967180 1895 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:42:59.967856 1890 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:42:59.970116 1896 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cec7f2500dd147f8be1c52680e0bff45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } } }
I20260808 23:42:59.970206 1897 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec7f2500dd147f8be1c52680e0bff45. Latest consensus state: current_term: 1 leader_uuid: "cec7f2500dd147f8be1c52680e0bff45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } } }
I20260808 23:42:59.970830 1896 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:59.970829 1897 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: This master's current role is: LEADER
I20260808 23:42:59.971788 1905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:42:59.976778 1905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:42:59.985263 1905 catalog_manager.cc:1347] Generated new cluster ID: cc4c5c74839f49ff90efd28c75149bd9
I20260808 23:42:59.985378 1905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:43:00.011369 1905 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:43:00.012921 1905 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:43:00.027671 1905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Generated new TSK 0
I20260808 23:43:00.028537 1905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:43:00.099929 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45339
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:00.436972 1917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:00.437278 1917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:00.437443 1917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:00.445583 1917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:00.445766 1917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:00.445959 1917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:43:00.458565 1917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:00.460472 1917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:00.461826 1917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:00.475333 1922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:00.475383 1925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:00.475299 1923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:00.476166 1917 server_base.cc:1034] running on GCE node
I20260808 23:43:00.476761 1917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:00.477664 1917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:00.478919 1917 hybrid_clock.cc:648] HybridClock initialized: now 1786232580478831 us; error 71 us; skew 500 ppm
I20260808 23:43:00.479231 1917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:00 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232580, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:00.482511 1917 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:43:00.484012 1917 webserver.cc:460] Webserver started at http://127.18.186.129:35729/ using document root <none> and password file <none>
I20260808 23:43:00.484673 1917 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:00.484838 1917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:00.485146 1917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:43:00.488186 1917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "103435e24f5542319eae7ee1bf435ac2"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "f1f5e0fdd370e956a9f102df5048f24b"
server_key_iv: "a3dd72fdff8479fdc6c6fa66fd95c495"
server_key_version: "encryptionkey@0"
I20260808 23:43:00.489235 1917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "103435e24f5542319eae7ee1bf435ac2"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "f1f5e0fdd370e956a9f102df5048f24b"
server_key_iv: "a3dd72fdff8479fdc6c6fa66fd95c495"
server_key_version: "encryptionkey@0"
I20260808 23:43:00.494304 1917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:43:00.497766 1933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:00.499042 1917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:43:00.499212 1917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "103435e24f5542319eae7ee1bf435ac2"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "f1f5e0fdd370e956a9f102df5048f24b"
server_key_iv: "a3dd72fdff8479fdc6c6fa66fd95c495"
server_key_version: "encryptionkey@0"
I20260808 23:43:00.499481 1917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:00.518764 1917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:00.522770 1917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:00.523106 1917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:00.524175 1917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:00.525545 1917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:00.525648 1917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:00.525801 1917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:00.525875 1917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:00.559847 1917 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:42715
I20260808 23:43:00.559896 2052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:42715 every 8 connection(s)
I20260808 23:43:00.561172 1917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:43:00.569474 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 1917
I20260808 23:43:00.569700 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:43:00.570461 19178 external_mini_cluster.cc:1428] Setting key dbdfcad7f95ac37c83db28f57a62d861
Aug 08 23:43:00 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232580, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:00.581816 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:00.563867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:42609 (local address 127.18.186.190:45339)
0808 23:43:00.564313 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:00.564318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:00.565309 (+ 991us) server_negotiation.cc:407] Connection header received
0808 23:43:00.567013 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:00.567017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:00.567081 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:00.567178 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:00.568743 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:00.569886 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:00.571763 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:00.572197 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:00.572319 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:00.576036 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:00.576041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:00.576044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:00.576087 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:00.578715 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:00.579502 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:00.579508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:00.579511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:00.579629 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:00.580074 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:00.580084 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:00.580086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:00.580345 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:00.580505 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:00.581207 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 23:43:00.581425 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":215,"threads_started":1}
I20260808 23:43:00.582737 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:00.564445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:42609)
0808 23:43:00.565148 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:00.565191 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:00.566771 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:00.567385 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:00.567393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:00.567959 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:00.568580 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:00.568600 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:00.570049 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:00.570053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:00.571582 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:00.571598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:00.572485 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:00.572494 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:00.572729 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:00.573399 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:00.573431 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:00.575819 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:00.578871 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:00.578879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:00.578895 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:00.579343 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:00.579744 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:00.579749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:00.579753 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:00.579943 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:00.580478 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:00.580487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:00.580825 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 23:43:00.581095 (+ 270us) client_negotiation.cc:240] Negotiation successful
0808 23:43:00.581620 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":273,"threads_started":1}
I20260808 23:43:00.584306 2054 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45339
I20260808 23:43:00.584632 2054 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:00.585533 2054 heartbeater.cc:511] Master 127.18.186.190:45339 requested a full tablet report, sending...
I20260808 23:43:00.587944 1850 ts_manager.cc:194] Registered new tserver with Master: 103435e24f5542319eae7ee1bf435ac2 (127.18.186.129:42715)
I20260808 23:43:00.589422 1850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:42609
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:43:00.631316 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45339
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:00.958484 2060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:00.958738 2060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:00.958848 2060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:00.965946 2060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:00.966055 2060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:00.966171 2060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:43:00.976203 2060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:00.977418 2060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:00.978708 2060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:00.992283 2065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:00.992317 2067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:00.992511 2069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:00.993069 2060 server_base.cc:1034] running on GCE node
I20260808 23:43:00.993626 2060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:00.994508 2060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:00.995688 2060 hybrid_clock.cc:648] HybridClock initialized: now 1786232580995657 us; error 49 us; skew 500 ppm
I20260808 23:43:00.995992 2060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:00 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232580, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:00.999094 2060 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:43:01.000654 2060 webserver.cc:460] Webserver started at http://127.18.186.130:38099/ using document root <none> and password file <none>
I20260808 23:43:01.001282 2060 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:01.001379 2060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:01.001622 2060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:43:01.004622 2060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "33e5fef2a9b9494090fd50ecda44d89b"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "cfa1dfc256102919fde278e04a067acf"
server_key_iv: "e31a3beb9ebf65c1764be68358cccdf5"
server_key_version: "encryptionkey@0"
I20260808 23:43:01.005400 2060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "33e5fef2a9b9494090fd50ecda44d89b"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "cfa1dfc256102919fde278e04a067acf"
server_key_iv: "e31a3beb9ebf65c1764be68358cccdf5"
server_key_version: "encryptionkey@0"
I20260808 23:43:01.010375 2060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:43:01.013814 2076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:01.015153 2060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:43:01.015323 2060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "33e5fef2a9b9494090fd50ecda44d89b"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "cfa1dfc256102919fde278e04a067acf"
server_key_iv: "e31a3beb9ebf65c1764be68358cccdf5"
server_key_version: "encryptionkey@0"
I20260808 23:43:01.015554 2060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:01.034653 2060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:01.038668 2060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:01.038954 2060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:01.039959 2060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:01.041224 2060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:01.041297 2060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:01.041497 2060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:01.041564 2060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:01.075284 2060 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:34637
I20260808 23:43:01.075331 2201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:34637 every 8 connection(s)
I20260808 23:43:01.076597 2060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:43:01.086964 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 2060
I20260808 23:43:01.087183 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:43:01.087819 19178 external_mini_cluster.cc:1428] Setting key e58bf5e87c3a0333d7c852ca602c50e5
Aug 08 23:43:01 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232580, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:01.096660 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.079077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:59131 (local address 127.18.186.190:45339)
0808 23:43:01.079262 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.079267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.080471 (+ 1204us) server_negotiation.cc:407] Connection header received
0808 23:43:01.082204 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.082221 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.082292 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.082380 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:01.084076 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.084785 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.086265 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.086673 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.086763 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.090884 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:01.090889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:01.090893 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:01.090924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:01.093559 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:01.094277 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:01.094285 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:01.094293 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:01.094418 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:01.094860 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:01.094865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:01.094867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:01.095144 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:01.095312 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.096059 (+ 747us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.096301 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:43:01.097678 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.079542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:59131)
0808 23:43:01.080302 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:01.080340 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:01.081945 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:01.082622 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:01.082630 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:01.083178 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:01.083920 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.083939 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.084925 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.084929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.086141 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.086153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.086907 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.086916 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.087238 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.087981 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:01.088022 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:01.090668 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:01.093712 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:01.093721 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:01.093736 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:01.094148 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:01.094538 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:01.094543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:01.094547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:01.094732 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:01.095274 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:01.095283 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:01.095685 (+ 402us) client_negotiation.cc:769] Sending connection context
0808 23:43:01.095954 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 23:43:01.096536 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":311,"threads_started":1}
I20260808 23:43:01.099166 2202 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45339
I20260808 23:43:01.099486 2202 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:01.100198 2202 heartbeater.cc:511] Master 127.18.186.190:45339 requested a full tablet report, sending...
I20260808 23:43:01.101909 1850 ts_manager.cc:194] Registered new tserver with Master: 33e5fef2a9b9494090fd50ecda44d89b (127.18.186.130:34637)
I20260808 23:43:01.102743 1850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:59131
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:43:01.150295 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45339
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:01.479604 2207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:01.479883 2207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:01.480036 2207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:01.487237 2207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:01.487366 2207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:01.487567 2207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:43:01.497653 2207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:01.498900 2207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:01.500164 2207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:01.513043 2215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:01.513011 2213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:01.513566 2212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:01.513985 2207 server_base.cc:1034] running on GCE node
I20260808 23:43:01.514633 2207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:01.515650 2207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:01.516876 2207 hybrid_clock.cc:648] HybridClock initialized: now 1786232581516822 us; error 46 us; skew 500 ppm
I20260808 23:43:01.517136 2207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:01 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:01.520395 2207 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:43:01.521754 2207 webserver.cc:460] Webserver started at http://127.18.186.131:39093/ using document root <none> and password file <none>
I20260808 23:43:01.522418 2207 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:01.522550 2207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:01.522866 2207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:43:01.525918 2207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "407c2eadf3d64a2e9f4406a0727b8d6c"
format_stamp: "Formatted at 2026-08-08 23:43:01 on dist-test-slave-5k54"
server_key: "fa04d879ecba39fdbbf3300f71a2e267"
server_key_iv: "36e32721262cf1ff17093f9ed5f96a78"
server_key_version: "encryptionkey@0"
I20260808 23:43:01.526705 2207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "407c2eadf3d64a2e9f4406a0727b8d6c"
format_stamp: "Formatted at 2026-08-08 23:43:01 on dist-test-slave-5k54"
server_key: "fa04d879ecba39fdbbf3300f71a2e267"
server_key_iv: "36e32721262cf1ff17093f9ed5f96a78"
server_key_version: "encryptionkey@0"
I20260808 23:43:01.531860 2207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260808 23:43:01.535221 2222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:01.536718 2207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:43:01.536911 2207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "407c2eadf3d64a2e9f4406a0727b8d6c"
format_stamp: "Formatted at 2026-08-08 23:43:01 on dist-test-slave-5k54"
server_key: "fa04d879ecba39fdbbf3300f71a2e267"
server_key_iv: "36e32721262cf1ff17093f9ed5f96a78"
server_key_version: "encryptionkey@0"
I20260808 23:43:01.537146 2207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:01.556258 2207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:01.560555 2207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:01.560891 2207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:01.562018 2207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:01.563748 2207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:01.563840 2207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:01.563953 2207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:01.564016 2207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:01.593178 2054 heartbeater.cc:503] Master 127.18.186.190:45339 was elected leader, sending a full tablet report...
I20260808 23:43:01.598583 2207 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:32965
I20260808 23:43:01.598623 2339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:32965 every 8 connection(s)
I20260808 23:43:01.599975 2207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:43:01.607290 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 2207
I20260808 23:43:01.607546 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:43:01.608184 19178 external_mini_cluster.cc:1428] Setting key d02ef253c69013d791d91a255b88c84d
Aug 08 23:43:01 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:01.620574 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.602778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:33447 (local address 127.18.186.190:45339)
0808 23:43:01.603235 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.603239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.604372 (+ 1133us) server_negotiation.cc:407] Connection header received
0808 23:43:01.606139 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.606144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.606209 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.606304 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:01.608211 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.609248 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.611296 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.611736 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.611831 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.615200 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:01.615204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:01.615208 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:01.615241 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:01.617607 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:01.618256 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:01.618259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:01.618262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:01.618380 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:01.618742 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:01.618745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:01.618747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:01.619002 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:01.619185 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.619973 (+ 788us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.620191 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":251,"threads_started":1}
I20260808 23:43:01.621495 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.603388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:33447)
0808 23:43:01.604208 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:01.604252 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:01.605879 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:01.606509 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:01.606517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:01.607165 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:01.608021 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.608047 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.609414 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.609421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.611135 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.611147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.611948 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.611954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.612145 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.612773 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:01.612811 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:01.614991 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:01.617729 (+ 2738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:01.617735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:01.617747 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:01.618157 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:01.618471 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:01.618474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:01.618476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:01.618651 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:01.619197 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:01.619207 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:01.619605 (+ 398us) client_negotiation.cc:769] Sending connection context
0808 23:43:01.619877 (+ 272us) client_negotiation.cc:240] Negotiation successful
0808 23:43:01.620405 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":325,"threads_started":1}
I20260808 23:43:01.623018 2340 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45339
I20260808 23:43:01.623353 2340 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:01.624099 2340 heartbeater.cc:511] Master 127.18.186.190:45339 requested a full tablet report, sending...
I20260808 23:43:01.625524 1848 ts_manager.cc:194] Registered new tserver with Master: 407c2eadf3d64a2e9f4406a0727b8d6c (127.18.186.131:32965)
I20260808 23:43:01.626410 1848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:33447
I20260808 23:43:01.635785 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:43:01.652680 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.640489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.18.186.190:45339)
0808 23:43:01.640669 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.640674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.641103 (+ 429us) server_negotiation.cc:407] Connection header received
0808 23:43:01.641241 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.641244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.641340 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.641459 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:01.642773 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.643564 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.644868 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.645310 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.645403 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.647713 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:01.647717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:01.647720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:01.647751 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:01.650379 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:01.651025 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:01.651029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:01.651031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:01.651147 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:01.651519 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:01.651523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:01.651525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:01.651840 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:01.652042 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.652284 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.652452 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:43:01.656747 1848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:43:01.663905 1848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:43:01.700759 2356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.677020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.18.186.131:32965)
0808 23:43:01.689560 (+ 12540us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.689571 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.690995 (+ 1424us) server_negotiation.cc:407] Connection header received
0808 23:43:01.691088 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.691094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.691307 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.691540 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:01.693038 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.694781 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.697835 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.699494 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.699726 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.699944 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.700153 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.700349 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12344,"thread_start_us":12555,"threads_started":1}
I20260808 23:43:01.701586 2354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.676723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:32965 (local address 127.0.0.1:44542)
0808 23:43:01.690108 (+ 13385us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:01.690150 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:01.690387 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:01.691852 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:01.691860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:01.692248 (+ 388us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:01.692753 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.692768 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.694920 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.694926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.697656 (+ 2730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.697669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.699683 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.699691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.699894 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.699947 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 23:43:01.700084 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 23:43:01.700259 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13116,"mutex_wait_us":43,"spinlock_wait_cycles":8448,"thread_start_us":12132,"threads_started":1}
I20260808 23:43:01.702082 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.677713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.18.186.129:42715)
0808 23:43:01.690520 (+ 12807us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.690527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.690802 (+ 275us) server_negotiation.cc:407] Connection header received
0808 23:43:01.691036 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.691045 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.691307 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.691540 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:01.693098 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.694782 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.698013 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.699610 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.699818 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.699962 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.701278 (+ 1316us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.701457 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12646,"thread_start_us":12473,"threads_started":1}
I20260808 23:43:01.702132 2360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.690864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:34637 (local address 127.0.0.1:37324)
0808 23:43:01.694461 (+ 3597us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:01.694476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:01.694622 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:01.695359 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:01.695364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:01.695396 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:01.695938 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.695953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.697914 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.697920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.699213 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.699224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.701358 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.701364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.701550 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.701593 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 23:43:01.701694 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 23:43:01.701811 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3457,"thread_start_us":3305,"threads_started":1}
I20260808 23:43:01.702596 2355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.676958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:42715 (local address 127.0.0.1:53622)
0808 23:43:01.690680 (+ 13722us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:01.690694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:01.690846 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:01.691859 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:01.691863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:01.691899 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:01.692891 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.692905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.694992 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.694997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.697853 (+ 2856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.697869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.700549 (+ 2680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.700561 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.700806 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.700852 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 23:43:01.700959 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 23:43:01.701090 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13627,"mutex_wait_us":125,"spinlock_wait_cycles":1013248,"thread_start_us":11972,"threads_started":1}
I20260808 23:43:01.702677 2358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.690081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.18.186.130:34637)
0808 23:43:01.694147 (+ 4066us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.694159 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.694598 (+ 439us) server_negotiation.cc:407] Connection header received
0808 23:43:01.694828 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.694835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.695132 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.695388 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:01.696162 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.697735 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.699375 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.700869 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.701028 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.701205 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.702083 (+ 878us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.702315 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3888,"thread_start_us":3747,"threads_started":1}
I20260808 23:43:01.705761 2273 tablet_service.cc:1467] Processing CreateTablet for tablet 22f9c2580b834e34904850f66aae74c2 (DEFAULT_TABLE table=test-table [id=5896768106e842d981d073886cf88527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:01.706614 2130 tablet_service.cc:1467] Processing CreateTablet for tablet 22f9c2580b834e34904850f66aae74c2 (DEFAULT_TABLE table=test-table [id=5896768106e842d981d073886cf88527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:01.706614 1985 tablet_service.cc:1467] Processing CreateTablet for tablet 22f9c2580b834e34904850f66aae74c2 (DEFAULT_TABLE table=test-table [id=5896768106e842d981d073886cf88527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:01.707384 2273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22f9c2580b834e34904850f66aae74c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:01.708175 1985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22f9c2580b834e34904850f66aae74c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:01.708173 2130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22f9c2580b834e34904850f66aae74c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:01.720050 2361 tablet_bootstrap.cc:492] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: Bootstrap starting.
I20260808 23:43:01.723104 2361 tablet_bootstrap.cc:654] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:01.724143 2363 tablet_bootstrap.cc:492] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Bootstrap starting.
I20260808 23:43:01.724509 2361 log.cc:824] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:01.725950 2362 tablet_bootstrap.cc:492] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Bootstrap starting.
I20260808 23:43:01.727921 2361 tablet_bootstrap.cc:492] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: No bootstrap required, opened a new log
I20260808 23:43:01.728008 2363 tablet_bootstrap.cc:654] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:01.728302 2361 ts_tablet_manager.cc:1397] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260808 23:43:01.729275 2363 log.cc:824] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:01.729523 2362 tablet_bootstrap.cc:654] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:01.730620 2362 log.cc:824] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:01.732347 2363 tablet_bootstrap.cc:492] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: No bootstrap required, opened a new log
I20260808 23:43:01.732623 2363 ts_tablet_manager.cc:1397] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 23:43:01.733636 2362 tablet_bootstrap.cc:492] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: No bootstrap required, opened a new log
I20260808 23:43:01.733886 2362 ts_tablet_manager.cc:1397] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260808 23:43:01.736486 2361 raft_consensus.cc:348] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.736850 2361 raft_consensus.cc:374] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:01.736972 2361 raft_consensus.cc:729] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407c2eadf3d64a2e9f4406a0727b8d6c, State: Initialized, Role: FOLLOWER
I20260808 23:43:01.737627 2361 consensus_queue.cc:260] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.742364 2340 heartbeater.cc:503] Master 127.18.186.190:45339 was elected leader, sending a full tablet report...
I20260808 23:43:01.742883 2361 ts_tablet_manager.cc:1428] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260808 23:43:01.743512 2363 raft_consensus.cc:348] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.743938 2363 raft_consensus.cc:374] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:01.744057 2363 raft_consensus.cc:729] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e5fef2a9b9494090fd50ecda44d89b, State: Initialized, Role: FOLLOWER
I20260808 23:43:01.744671 2363 consensus_queue.cc:260] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.745553 2362 raft_consensus.cc:348] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.745781 2202 heartbeater.cc:503] Master 127.18.186.190:45339 was elected leader, sending a full tablet report...
I20260808 23:43:01.745949 2362 raft_consensus.cc:374] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:01.746061 2362 raft_consensus.cc:729] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103435e24f5542319eae7ee1bf435ac2, State: Initialized, Role: FOLLOWER
I20260808 23:43:01.746681 2362 consensus_queue.cc:260] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.747452 2363 ts_tablet_manager.cc:1428] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260808 23:43:01.748986 2362 ts_tablet_manager.cc:1428] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260808 23:43:01.774129 2368 raft_consensus.cc:482] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:43:01.774461 2368 raft_consensus.cc:504] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.776317 2368 leader_election.cc:288] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 103435e24f5542319eae7ee1bf435ac2 (127.18.186.129:42715), 33e5fef2a9b9494090fd50ecda44d89b (127.18.186.130:34637)
I20260808 23:43:01.783354 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.776489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:42715 (local address 127.18.186.131:60929)
0808 23:43:01.776770 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:01.776793 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:01.776929 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:01.777446 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:01.777453 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:01.777483 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:01.777712 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.777726 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.779135 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.779139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.781173 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.781187 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.782654 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.782659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.782815 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.782839 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:43:01.782923 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 23:43:01.783012 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":44}
I20260808 23:43:01.783357 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.776754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:60929 (local address 127.18.186.129:42715)
0808 23:43:01.777024 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.777034 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.777064 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:43:01.777169 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.777179 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.777281 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.777403 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:01.777864 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.778998 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.781366 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.782485 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.782660 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.782737 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.782979 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.783121 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 23:43:01.784054 2358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.777007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:37603 (local address 127.18.186.130:34637)
0808 23:43:01.777237 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.777244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.777666 (+ 422us) server_negotiation.cc:407] Connection header received
0808 23:43:01.777848 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.777854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.777943 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.778049 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:01.778496 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.779572 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.781826 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.783019 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.783160 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.783225 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.783670 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.783777 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:43:01.784415 2005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22f9c2580b834e34904850f66aae74c2" candidate_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103435e24f5542319eae7ee1bf435ac2" is_pre_election: true
I20260808 23:43:01.784804 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.776755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:34637 (local address 127.18.186.131:37603)
0808 23:43:01.777543 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:01.777566 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:01.777687 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:01.778069 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:01.778073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:01.778093 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:01.778342 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.778349 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.779709 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.779714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.781663 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:01.781674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.783234 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.783240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:01.783462 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.783504 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 23:43:01.784239 (+ 735us) client_negotiation.cc:240] Negotiation successful
0808 23:43:01.784381 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":22,"thread_start_us":402,"threads_started":1}
I20260808 23:43:01.785058 2005 raft_consensus.cc:2454] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407c2eadf3d64a2e9f4406a0727b8d6c in term 0.
I20260808 23:43:01.785753 2223 leader_election.cc:302] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103435e24f5542319eae7ee1bf435ac2, 407c2eadf3d64a2e9f4406a0727b8d6c; no voters:
I20260808 23:43:01.785905 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22f9c2580b834e34904850f66aae74c2" candidate_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33e5fef2a9b9494090fd50ecda44d89b" is_pre_election: true
I20260808 23:43:01.786201 2368 raft_consensus.cc:2781] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:43:01.786362 2368 raft_consensus.cc:482] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:43:01.786442 2368 raft_consensus.cc:3037] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:01.786418 2150 raft_consensus.cc:2454] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407c2eadf3d64a2e9f4406a0727b8d6c in term 0.
I20260808 23:43:01.789108 2368 raft_consensus.cc:504] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.789754 2368 leader_election.cc:288] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [CANDIDATE]: Term 1 election: Requested vote from peers 103435e24f5542319eae7ee1bf435ac2 (127.18.186.129:42715), 33e5fef2a9b9494090fd50ecda44d89b (127.18.186.130:34637)
I20260808 23:43:01.790125 2005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22f9c2580b834e34904850f66aae74c2" candidate_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "103435e24f5542319eae7ee1bf435ac2"
I20260808 23:43:01.790226 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22f9c2580b834e34904850f66aae74c2" candidate_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33e5fef2a9b9494090fd50ecda44d89b"
I20260808 23:43:01.790354 2005 raft_consensus.cc:3037] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:01.790436 2150 raft_consensus.cc:3037] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:01.793656 2005 raft_consensus.cc:2454] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407c2eadf3d64a2e9f4406a0727b8d6c in term 1.
I20260808 23:43:01.793656 2150 raft_consensus.cc:2454] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407c2eadf3d64a2e9f4406a0727b8d6c in term 1.
I20260808 23:43:01.794097 2226 leader_election.cc:302] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33e5fef2a9b9494090fd50ecda44d89b, 407c2eadf3d64a2e9f4406a0727b8d6c; no voters:
I20260808 23:43:01.794423 2368 raft_consensus.cc:2781] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:43:01.794922 2368 raft_consensus.cc:686] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 LEADER]: Becoming Leader. State: Replica: 407c2eadf3d64a2e9f4406a0727b8d6c, State: Running, Role: LEADER
I20260808 23:43:01.795403 2368 consensus_queue.cc:237] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } }
I20260808 23:43:01.800190 1848 catalog_manager.cc:5495] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 407c2eadf3d64a2e9f4406a0727b8d6c (127.18.186.131). New cstate: current_term: 1 leader_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } health_report { overall_health: UNKNOWN } } }
W20260808 23:43:01.813697 2055 tablet.cc:2369] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:43:01.823259 2356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.817911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.18.186.131:32965)
0808 23:43:01.818126 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.818134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.818241 (+ 107us) server_negotiation.cc:407] Connection header received
0808 23:43:01.818407 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.818410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.818487 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.818663 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:43:01.819009 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.819902 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.821279 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.821676 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.821768 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.822225 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:43:01.822724 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:43:01.822838 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.822910 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.822972 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 23:43:01.828475 2203 tablet.cc:2369] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:43:01.831125 2150 raft_consensus.cc:1261] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 1 FOLLOWER]: Refusing update from remote peer 407c2eadf3d64a2e9f4406a0727b8d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:43:01.831167 2005 raft_consensus.cc:1261] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 1 FOLLOWER]: Refusing update from remote peer 407c2eadf3d64a2e9f4406a0727b8d6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:43:01.832157 2373 consensus_queue.cc:1035] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:43:01.832499 2368 consensus_queue.cc:1035] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:43:01.865461 1850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:43:01.865697 1850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:43:01.869551 1850 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Sending DeleteTablet for 3 replicas of tablet 22f9c2580b834e34904850f66aae74c2
I20260808 23:43:01.870788 1985 tablet_service.cc:1514] Processing DeleteTablet for tablet 22f9c2580b834e34904850f66aae74c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:01 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53622
I20260808 23:43:01.870882 2130 tablet_service.cc:1514] Processing DeleteTablet for tablet 22f9c2580b834e34904850f66aae74c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:01 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:37324
I20260808 23:43:01.871310 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 1917
I20260808 23:43:01.871711 2273 tablet_service.cc:1514] Processing DeleteTablet for tablet 22f9c2580b834e34904850f66aae74c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:01 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:44542
I20260808 23:43:01.882793 2387 tablet_replica.cc:330] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: stopping tablet replica
I20260808 23:43:01.883466 2387 raft_consensus.cc:2229] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:43:01.884133 2387 raft_consensus.cc:2258] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:01.886579 2390 tablet_replica.cc:330] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: stopping tablet replica
I20260808 23:43:01.887225 2390 raft_consensus.cc:2229] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:43:01.888053 2390 raft_consensus.cc:2258] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:01.890234 2390 ts_tablet_manager.cc:1905] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:01.890228 2387 ts_tablet_manager.cc:1905] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:01.895965 2388 tablet_replica.cc:330] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: stopping tablet replica
I20260808 23:43:01.896426 2388 raft_consensus.cc:2229] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:43:01.896946 2388 raft_consensus.cc:2258] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:01.897473 2387 ts_tablet_manager.cc:1918] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:01.897650 2387 log.cc:1196] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/22f9c2580b834e34904850f66aae74c2
I20260808 23:43:01.898201 2387 ts_tablet_manager.cc:1939] T 22f9c2580b834e34904850f66aae74c2 P 103435e24f5542319eae7ee1bf435ac2: Deleting consensus metadata
I20260808 23:43:01.899811 2390 ts_tablet_manager.cc:1918] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:01.900341 1832 catalog_manager.cc:4841] TS 103435e24f5542319eae7ee1bf435ac2 (127.18.186.129:42715): tablet 22f9c2580b834e34904850f66aae74c2 (table test-table [id=5896768106e842d981d073886cf88527]) successfully deleted
I20260808 23:43:01.900846 2390 log.cc:1196] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/22f9c2580b834e34904850f66aae74c2
I20260808 23:43:01.901579 2390 ts_tablet_manager.cc:1939] T 22f9c2580b834e34904850f66aae74c2 P 407c2eadf3d64a2e9f4406a0727b8d6c: Deleting consensus metadata
I20260808 23:43:01.902163 2388 ts_tablet_manager.cc:1905] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:01.903535 1835 catalog_manager.cc:4841] TS 407c2eadf3d64a2e9f4406a0727b8d6c (127.18.186.131:32965): tablet 22f9c2580b834e34904850f66aae74c2 (table test-table [id=5896768106e842d981d073886cf88527]) successfully deleted
Aug 08 23:43:01 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:43:01.910009 2388 ts_tablet_manager.cc:1918] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:01.910180 2388 log.cc:1196] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/22f9c2580b834e34904850f66aae74c2
I20260808 23:43:01.910739 2388 ts_tablet_manager.cc:1939] T 22f9c2580b834e34904850f66aae74c2 P 33e5fef2a9b9494090fd50ecda44d89b: Deleting consensus metadata
I20260808 23:43:01.912453 1835 catalog_manager.cc:4841] TS 33e5fef2a9b9494090fd50ecda44d89b (127.18.186.130:34637): tablet 22f9c2580b834e34904850f66aae74c2 (table test-table [id=5896768106e842d981d073886cf88527]) successfully deleted
I20260808 23:43:01.916798 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:01.872750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53634 (local address 127.18.186.129:42715)
0808 23:43:01.872964 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:01.872973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:01.885271 (+ 12298us) server_negotiation.cc:407] Connection header received
0808 23:43:01.885930 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:01.885938 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:01.886044 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:01.886141 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:01.899598 (+ 13457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.900847 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.902940 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:01.903601 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:01.903730 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:01.909458 (+ 5728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:01.909477 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:01.909496 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:01.909563 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:01.913703 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:01.914478 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:01.914488 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:01.914492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:01.914673 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:01.915051 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:01.915058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:01.915063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:01.915618 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:01.915944 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:01.916140 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 23:43:01.916367 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:43:01.917541 2047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:02.008579 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 1917
I20260808 23:43:02.028748 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 2060
Aug 08 23:43:02 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:43:02.045543 2358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:02.029697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37326 (local address 127.18.186.130:34637)
0808 23:43:02.029892 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:02.029899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:02.029922 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:43:02.030008 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:02.030013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:02.030102 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:02.030207 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:02.031512 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:02.032292 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:02.033527 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:02.033932 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:02.034061 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:02.039123 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:02.039136 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:02.039151 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:02.039189 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:02.042486 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:02.043380 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:02.043387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:02.043390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:02.043687 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:02.044110 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:02.044113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:02.044115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:02.044476 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:02.044701 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:02.045078 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 23:43:02.045240 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:43:02.046295 2197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:02.136776 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 2060
I20260808 23:43:02.156380 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 2207
Aug 08 23:43:02 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:43:02.176319 2356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:02.157363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44552 (local address 127.18.186.131:32965)
0808 23:43:02.157540 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:02.157545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:02.157563 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:43:02.157759 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:02.157763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:02.157834 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:02.157913 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:02.159959 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:02.160780 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:02.162747 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:02.163284 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:02.163390 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:02.169313 (+ 5923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:02.169328 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:02.169350 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:02.169392 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:02.172953 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:02.173980 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:02.173988 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:02.173990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:02.174132 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:02.174582 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:02.174585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:02.174587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:02.174940 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:02.175168 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:02.175708 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 23:43:02.175878 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:43:02.177055 2335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:02.303416 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 2207
I20260808 23:43:02.322379 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 1815
I20260808 23:43:02.323611 1885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:02.435575 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 1815
I20260808 23:43:02.461798 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:45339
--webserver_interface=127.18.186.190
--webserver_port=33637
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:45339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:02.800757 2397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:02.801153 2397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:02.801321 2397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:02.808606 2397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:43:02.808759 2397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:02.808882 2397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:02.808979 2397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:43:02.809083 2397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:43:02.818965 2397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:45339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:45339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=33637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:02.820320 2397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:02.821630 2397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:02.847922 2406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:02.847991 2403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:02.847961 2404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:02.849243 2397 server_base.cc:1034] running on GCE node
I20260808 23:43:02.850199 2397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:02.851955 2397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:02.853199 2397 hybrid_clock.cc:648] HybridClock initialized: now 1786232582853141 us; error 68 us; skew 500 ppm
I20260808 23:43:02.853627 2397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:02 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232582, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:02.857267 2397 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:43:02.859020 2397 webserver.cc:460] Webserver started at http://127.18.186.190:33637/ using document root <none> and password file <none>
I20260808 23:43:02.859825 2397 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:02.859961 2397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:02.865954 2397 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:43:02.868888 2415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:02.870714 2397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:43:02.870918 2397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "cec7f2500dd147f8be1c52680e0bff45"
format_stamp: "Formatted at 2026-08-08 23:42:59 on dist-test-slave-5k54"
server_key: "2c7aa51a13e5e3714441f98eb6dd96b5"
server_key_iv: "1ffe36b418bed15ed21d4e09a3f9181f"
server_key_version: "encryptionkey@0"
I20260808 23:43:02.872646 2397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:02.908584 2397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:02.912847 2397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:02.913156 2397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:02.933004 2397 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:45339
I20260808 23:43:02.933033 2472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:45339 every 8 connection(s)
I20260808 23:43:02.934569 2397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:43:02.939718 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 2397
I20260808 23:43:02.940766 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:42715
--local_ip_for_outbound_sockets=127.18.186.129
--tserver_master_addrs=127.18.186.190:45339
--webserver_port=35729
--webserver_interface=127.18.186.129
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:43:02.942919 2473 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:43:02.948331 2473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Bootstrap starting.
I20260808 23:43:02.969609 2473 log.cc:824] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:02.983291 2473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:43:02.983987 2473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Bootstrap complete.
I20260808 23:43:02.993569 2473 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:43:02.994443 2473 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec7f2500dd147f8be1c52680e0bff45, State: Initialized, Role: FOLLOWER
I20260808 23:43:02.995139 2473 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:43:02.995383 2473 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:43:02.995525 2473 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:43:02.995708 2473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:43:02.999606 2473 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:43:03.000152 2473 leader_election.cc:302] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec7f2500dd147f8be1c52680e0bff45; no voters:
I20260808 23:43:03.000830 2473 leader_election.cc:288] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:43:03.000960 2477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:43:03.002216 2477 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 2 LEADER]: Becoming Leader. State: Replica: cec7f2500dd147f8be1c52680e0bff45, State: Running, Role: LEADER
I20260808 23:43:03.002913 2477 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:43:03.003456 2473 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:43:03.006142 2479 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec7f2500dd147f8be1c52680e0bff45. Latest consensus state: current_term: 2 leader_uuid: "cec7f2500dd147f8be1c52680e0bff45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } } }
I20260808 23:43:03.006177 2478 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cec7f2500dd147f8be1c52680e0bff45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } } }
I20260808 23:43:03.006532 2479 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: This master's current role is: LEADER
I20260808 23:43:03.006609 2478 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: This master's current role is: LEADER
I20260808 23:43:03.016788 2484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:43:03.026741 2484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:43:03.028699 2484 catalog_manager.cc:1259] Loaded cluster ID: cc4c5c74839f49ff90efd28c75149bd9
I20260808 23:43:03.028818 2484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:43:03.032827 2484 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 23:43:03.053139 2484 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 23:43:03.061288 2484 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:43:03.062640 2484 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Loaded TSK: 0
I20260808 23:43:03.063946 2484 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:43:03.318789 2475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:03.319059 2475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:03.319204 2475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:03.326635 2475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:03.326745 2475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:03.326920 2475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:43:03.337554 2475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:42715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=35729
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:03.339087 2475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:03.340517 2475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:03.354382 2502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:03.354884 2501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:03.354913 2504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:03.355329 2475 server_base.cc:1034] running on GCE node
I20260808 23:43:03.355959 2475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:03.356953 2475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:03.358170 2475 hybrid_clock.cc:648] HybridClock initialized: now 1786232583358117 us; error 49 us; skew 500 ppm
I20260808 23:43:03.358489 2475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:03 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232583, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:03.366564 2475 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:43:03.368309 2475 webserver.cc:460] Webserver started at http://127.18.186.129:35729/ using document root <none> and password file <none>
I20260808 23:43:03.369167 2475 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:03.369313 2475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:03.376099 2475 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 23:43:03.379549 2511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:03.381268 2475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:43:03.381518 2475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "103435e24f5542319eae7ee1bf435ac2"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "f1f5e0fdd370e956a9f102df5048f24b"
server_key_iv: "a3dd72fdff8479fdc6c6fa66fd95c495"
server_key_version: "encryptionkey@0"
I20260808 23:43:03.382678 2475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:03.407361 2475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:03.411749 2475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:03.412029 2475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:03.413021 2475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:03.414330 2475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:03.414463 2475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:03.414594 2475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:03.414680 2475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:03.448280 2475 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:42715
I20260808 23:43:03.448370 2630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:42715 every 8 connection(s)
I20260808 23:43:03.449771 2475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:43:03.450896 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 2475
I20260808 23:43:03.451748 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:34637
--local_ip_for_outbound_sockets=127.18.186.130
--tserver_master_addrs=127.18.186.190:45339
--webserver_port=38099
--webserver_interface=127.18.186.130
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:43:03 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232583, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:03.477210 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:03.453348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:44795)
0808 23:43:03.454318 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:03.454367 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:03.457096 (+ 2729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:03.458530 (+ 1434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:03.458544 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:03.459323 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:03.460169 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:03.460196 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:03.461876 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:03.461884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:03.463162 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:03.463175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:03.464705 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:03.464710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:03.464920 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:03.465623 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:03.465645 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:03.468079 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:03.472898 (+ 4819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:03.472905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:03.472916 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:03.473349 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:03.473872 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:03.473875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:03.473877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:03.474061 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:03.475042 (+ 981us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:03.475047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:03.475368 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 23:43:03.475692 (+ 324us) client_negotiation.cc:240] Negotiation successful
0808 23:43:03.476099 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":416,"threads_started":1}
I20260808 23:43:03.477929 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:03.453216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:44795 (local address 127.18.186.190:45339)
0808 23:43:03.453966 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:03.453984 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:03.454509 (+ 525us) server_negotiation.cc:407] Connection header received
0808 23:43:03.457638 (+ 3129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:03.457652 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:03.457991 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:03.458293 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:03.460412 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:03.461693 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:03.463320 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:03.464485 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:03.464694 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:03.468364 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:03.468377 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:03.468397 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:03.468449 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:03.472757 (+ 4308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:03.473518 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:03.473530 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:03.473541 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:03.473762 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:03.474188 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:03.474196 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:03.474202 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:03.474903 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:03.475133 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:03.475993 (+ 860us) server_negotiation.cc:299] Negotiation successful
0808 23:43:03.476491 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":306,"threads_started":1}
I20260808 23:43:03.480906 2631 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45339
I20260808 23:43:03.481230 2631 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:03.481953 2631 heartbeater.cc:511] Master 127.18.186.190:45339 requested a full tablet report, sending...
I20260808 23:43:03.484145 2435 ts_manager.cc:194] Registered new tserver with Master: 103435e24f5542319eae7ee1bf435ac2 (127.18.186.129:42715)
I20260808 23:43:03.485617 2435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:44795
W20260808 23:43:03.786235 2635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:03.786589 2635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:03.786767 2635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:03.794011 2635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:03.794111 2635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:03.794258 2635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:43:03.803937 2635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:34637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=38099
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:03.805339 2635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:03.806684 2635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:03.820232 2644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:03.820536 2640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:03.820276 2642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:03.820940 2635 server_base.cc:1034] running on GCE node
I20260808 23:43:03.821735 2635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:03.822717 2635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:03.823913 2635 hybrid_clock.cc:648] HybridClock initialized: now 1786232583823849 us; error 54 us; skew 500 ppm
I20260808 23:43:03.824191 2635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:03 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232583, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:03.827281 2635 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:43:03.828748 2635 webserver.cc:460] Webserver started at http://127.18.186.130:38099/ using document root <none> and password file <none>
I20260808 23:43:03.829459 2635 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:03.829614 2635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:03.835050 2635 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:43:03.838025 2651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:03.839362 2635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:43:03.839603 2635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "33e5fef2a9b9494090fd50ecda44d89b"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "cfa1dfc256102919fde278e04a067acf"
server_key_iv: "e31a3beb9ebf65c1764be68358cccdf5"
server_key_version: "encryptionkey@0"
I20260808 23:43:03.840567 2635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:03.858062 2635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:03.862069 2635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:03.862337 2635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:03.863328 2635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:03.864660 2635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:03.864794 2635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:03.864929 2635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:03.865024 2635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:03.898916 2635 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:34637
I20260808 23:43:03.899011 2774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:34637 every 8 connection(s)
I20260808 23:43:03.900504 2635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:43:03.903642 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 2635
I20260808 23:43:03.904484 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:32965
--local_ip_for_outbound_sockets=127.18.186.131
--tserver_master_addrs=127.18.186.190:45339
--webserver_port=39093
--webserver_interface=127.18.186.131
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:43:03 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232583, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:03.927713 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:03.903078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:38301 (local address 127.18.186.190:45339)
0808 23:43:03.903350 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:03.903357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:03.904668 (+ 1311us) server_negotiation.cc:407] Connection header received
0808 23:43:03.907697 (+ 3029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:03.907702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:03.907775 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:03.907857 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:03.910066 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:03.910739 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:03.913402 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:03.914160 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:03.914304 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:03.920113 (+ 5809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:03.920122 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:03.920130 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:03.920178 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:03.924110 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:03.924873 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:03.924881 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:03.924887 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:03.925083 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:03.925544 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:03.925552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:03.925558 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:03.926018 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:03.926179 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:03.926884 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 23:43:03.927165 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260808 23:43:03.928274 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:03.903591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:38301)
0808 23:43:03.904471 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:03.904547 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:03.907341 (+ 2794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:03.908148 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:03.908162 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:03.908957 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:03.909798 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:03.909826 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:03.910918 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:03.910926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:03.913158 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:03.913176 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:03.914643 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:03.914653 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:03.915006 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:03.915844 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:03.915867 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:03.919846 (+ 3979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:03.924255 (+ 4409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:03.924263 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:03.924276 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:03.924707 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:03.925220 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:03.925224 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:03.925226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:03.925407 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:03.926141 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:03.926147 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:03.926486 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 23:43:03.926715 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 23:43:03.927119 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":386,"threads_started":1}
I20260808 23:43:03.929809 2775 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45339
I20260808 23:43:03.930158 2775 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:03.931110 2775 heartbeater.cc:511] Master 127.18.186.190:45339 requested a full tablet report, sending...
I20260808 23:43:03.932966 2435 ts_manager.cc:194] Registered new tserver with Master: 33e5fef2a9b9494090fd50ecda44d89b (127.18.186.130:34637)
I20260808 23:43:03.934095 2435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:38301
W20260808 23:43:04.255251 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:04.255647 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:04.255834 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:04.263257 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:04.263417 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:04.263645 2779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:43:04.273505 2779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:32965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=39093
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:04.274796 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:04.276180 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:04.289866 2786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:04.289983 2785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:04.289996 2790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:04.291718 2779 server_base.cc:1034] running on GCE node
I20260808 23:43:04.292387 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:04.293319 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:04.294540 2779 hybrid_clock.cc:648] HybridClock initialized: now 1786232584294482 us; error 56 us; skew 500 ppm
I20260808 23:43:04.294842 2779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:04 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:04.298147 2779 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:43:04.299561 2779 webserver.cc:460] Webserver started at http://127.18.186.131:39093/ using document root <none> and password file <none>
I20260808 23:43:04.300186 2779 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:04.300302 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:04.305510 2779 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:43:04.308094 2797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:04.309360 2779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:43:04.309504 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "407c2eadf3d64a2e9f4406a0727b8d6c"
format_stamp: "Formatted at 2026-08-08 23:43:01 on dist-test-slave-5k54"
server_key: "fa04d879ecba39fdbbf3300f71a2e267"
server_key_iv: "36e32721262cf1ff17093f9ed5f96a78"
server_key_version: "encryptionkey@0"
I20260808 23:43:04.310382 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:04.356603 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:04.361042 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:04.361307 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:04.362267 2779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:04.363570 2779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:04.363683 2779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:04.363767 2779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:04.363816 2779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:04.397039 2779 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:32965
I20260808 23:43:04.397127 2920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:32965 every 8 connection(s)
I20260808 23:43:04.398509 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:43:04.400101 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 2779
Aug 08 23:43:04 dist-test-slave-5k54 krb5kdc[1801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:04.415992 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.401240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.18.186.190:45339)
0808 23:43:04.401438 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.401446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.402046 (+ 600us) server_negotiation.cc:407] Connection header received
0808 23:43:04.402275 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.402280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.402358 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.402472 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:04.404681 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.405490 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.406851 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.407369 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.407582 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.410339 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:04.410345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:04.410349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:04.410391 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:04.413125 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.413878 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.413883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.413887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.414042 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.414487 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.414494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.414508 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.414889 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:04.415241 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.415552 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.415715 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 23:43:04.421211 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.401259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:43149 (local address 127.18.186.190:45339)
0808 23:43:04.403816 (+ 2557us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.403824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.403846 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:43:04.405268 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.405273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.405360 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.405490 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:04.406990 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.407965 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.409491 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.409908 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.410038 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.413707 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:04.413712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:04.413715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:04.413745 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:04.417749 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.418909 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.418912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.418914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.419024 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.419477 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.419482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.419485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.419743 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:04.419895 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.420601 (+ 706us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.420805 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":205,"server-negotiator.queue_time_us":2445,"thread_start_us":2082,"threads_started":1}
I20260808 23:43:04.422180 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.401986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:43149)
0808 23:43:04.402897 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:04.402953 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:04.405014 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:04.405741 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:04.405751 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:04.406247 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:04.406798 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.406815 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.408124 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.408131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.409339 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.409351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.410144 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.410151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.410377 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.411033 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:04.411068 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:04.413487 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:04.417945 (+ 4458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:04.417959 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:04.417972 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:04.418635 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:04.419139 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:04.419144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:04.419147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:04.419339 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:04.419887 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:04.419895 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:04.420250 (+ 355us) client_negotiation.cc:769] Sending connection context
0808 23:43:04.420515 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 23:43:04.421075 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":427,"threads_started":1}
I20260808 23:43:04.423529 2923 heartbeater.cc:347] Connected to a master server at 127.18.186.190:45339
I20260808 23:43:04.423803 2923 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:04.424477 2923 heartbeater.cc:511] Master 127.18.186.190:45339 requested a full tablet report, sending...
I20260808 23:43:04.425823 2435 ts_manager.cc:194] Registered new tserver with Master: 407c2eadf3d64a2e9f4406a0727b8d6c (127.18.186.131:32965)
I20260808 23:43:04.426580 2435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:43149
I20260808 23:43:04.430799 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:43:04.437248 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.431655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.18.186.190:45339)
0808 23:43:04.431852 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.431859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.432341 (+ 482us) server_negotiation.cc:407] Connection header received
0808 23:43:04.432483 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.432487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.432619 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.432731 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:43:04.433132 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.433807 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.435195 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.435681 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.435778 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.436223 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:43:04.436684 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:43:04.436786 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.436874 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.436933 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:43:04.438058 2435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:43:04.445968 2435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:43:04.478451 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.458855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43094 (local address 127.18.186.129:42715)
0808 23:43:04.466910 (+ 8055us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.466922 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.467639 (+ 717us) server_negotiation.cc:407] Connection header received
0808 23:43:04.467770 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.467782 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.468097 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.469686 (+ 1589us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:04.470853 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.472739 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.474900 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.476439 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.476636 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.476817 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.477925 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.478126 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7825,"thread_start_us":7667,"threads_started":1}
I20260808 23:43:04.479053 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.458083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:42715 (local address 127.0.0.1:43094)
0808 23:43:04.467347 (+ 9264us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:04.467365 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:04.467503 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:04.470045 (+ 2542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:04.470056 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:04.470099 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:04.470600 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.470617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.472933 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.472938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.474635 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.474647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.476977 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.476984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.477261 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.477322 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 23:43:04.477505 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 23:43:04.477699 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9133,"mutex_wait_us":58,"thread_start_us":6501,"threads_started":1}
I20260808 23:43:04.480443 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.457873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.18.186.130:34637)
0808 23:43:04.471817 (+ 13944us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.471826 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.471855 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:43:04.471969 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.471977 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.472221 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.472404 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:04.473339 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.474996 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.477590 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.478965 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.479176 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.479343 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.479943 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.480107 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13793,"thread_start_us":13656,"threads_started":1}
I20260808 23:43:04.480443 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.457652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:34637 (local address 127.0.0.1:54782)
0808 23:43:04.465122 (+ 7470us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:04.465180 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:04.465407 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:04.472619 (+ 7212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:04.472624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:04.472651 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:04.473098 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.473111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.475149 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.475155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.477428 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.477443 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.479576 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.479582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.479761 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.479802 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 23:43:04.479897 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 23:43:04.480040 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7132,"mutex_wait_us":34,"thread_start_us":9442,"threads_started":1}
I20260808 23:43:04.483115 2562 tablet_service.cc:1467] Processing CreateTablet for tablet 5b01756d488a431da7ef1833ae011e77 (DEFAULT_TABLE table=test-table [id=28a85acce8f04c2e97f83a4329f27227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:04.483821 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.467496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:32965 (local address 127.0.0.1:53494)
0808 23:43:04.471024 (+ 3528us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:04.471043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:04.471150 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:04.473016 (+ 1866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:04.473025 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:04.473053 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:04.473584 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.473602 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.475672 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.475682 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.477701 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.477715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.479675 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.479680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.479834 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.479876 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 23:43:04.479949 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:43:04.480070 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3370,"mutex_wait_us":9,"thread_start_us":3731,"threads_started":1}
I20260808 23:43:04.483780 2708 tablet_service.cc:1467] Processing CreateTablet for tablet 5b01756d488a431da7ef1833ae011e77 (DEFAULT_TABLE table=test-table [id=28a85acce8f04c2e97f83a4329f27227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:04.484606 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.467995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.18.186.131:32965)
0808 23:43:04.472308 (+ 4313us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.472317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.472351 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:43:04.472459 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.472469 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.472779 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.473807 (+ 1028us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:04.474093 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.475546 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.477866 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.479470 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.479672 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.479825 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.480220 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.484253 (+ 4033us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4143,"thread_start_us":3969,"threads_started":1}
I20260808 23:43:04.484750 2562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b01756d488a431da7ef1833ae011e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:04.485070 2708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b01756d488a431da7ef1833ae011e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:04.488730 2631 heartbeater.cc:503] Master 127.18.186.190:45339 was elected leader, sending a full tablet report...
I20260808 23:43:04.497558 2941 tablet_bootstrap.cc:492] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Bootstrap starting.
I20260808 23:43:04.498533 2942 tablet_bootstrap.cc:492] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: Bootstrap starting.
I20260808 23:43:04.498657 2847 tablet_service.cc:1467] Processing CreateTablet for tablet 5b01756d488a431da7ef1833ae011e77 (DEFAULT_TABLE table=test-table [id=28a85acce8f04c2e97f83a4329f27227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:04.500080 2847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b01756d488a431da7ef1833ae011e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:04.502331 2942 tablet_bootstrap.cc:654] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:04.502332 2941 tablet_bootstrap.cc:654] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:04.503667 2942 log.cc:824] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:04.503667 2941 log.cc:824] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:04.506695 2942 tablet_bootstrap.cc:492] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: No bootstrap required, opened a new log
I20260808 23:43:04.506978 2942 ts_tablet_manager.cc:1397] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 23:43:04.507217 2941 tablet_bootstrap.cc:492] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: No bootstrap required, opened a new log
I20260808 23:43:04.507517 2941 ts_tablet_manager.cc:1397] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260808 23:43:04.514254 2942 raft_consensus.cc:348] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.514791 2942 raft_consensus.cc:374] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:04.514940 2942 raft_consensus.cc:729] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103435e24f5542319eae7ee1bf435ac2, State: Initialized, Role: FOLLOWER
I20260808 23:43:04.515754 2942 consensus_queue.cc:260] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.516396 2945 tablet_bootstrap.cc:492] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Bootstrap starting.
I20260808 23:43:04.518069 2942 ts_tablet_manager.cc:1428] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260808 23:43:04.518610 2941 raft_consensus.cc:348] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.519160 2941 raft_consensus.cc:374] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:04.519269 2941 raft_consensus.cc:729] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e5fef2a9b9494090fd50ecda44d89b, State: Initialized, Role: FOLLOWER
I20260808 23:43:04.519899 2941 consensus_queue.cc:260] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.520982 2945 tablet_bootstrap.cc:654] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:04.521121 2775 heartbeater.cc:503] Master 127.18.186.190:45339 was elected leader, sending a full tablet report...
I20260808 23:43:04.521795 2941 ts_tablet_manager.cc:1428] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 23:43:04.522400 2945 log.cc:824] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:04.525907 2945 tablet_bootstrap.cc:492] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: No bootstrap required, opened a new log
I20260808 23:43:04.526183 2945 ts_tablet_manager.cc:1397] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260808 23:43:04.533388 2945 raft_consensus.cc:348] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.533710 2945 raft_consensus.cc:374] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:04.533846 2945 raft_consensus.cc:729] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407c2eadf3d64a2e9f4406a0727b8d6c, State: Initialized, Role: FOLLOWER
I20260808 23:43:04.534413 2945 consensus_queue.cc:260] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.535389 2923 heartbeater.cc:503] Master 127.18.186.190:45339 was elected leader, sending a full tablet report...
I20260808 23:43:04.535866 2945 ts_tablet_manager.cc:1428] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Time spent starting tablet: real 0.010s user 0.004s sys 0.007s
I20260808 23:43:04.593060 2946 raft_consensus.cc:482] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:43:04.593353 2946 raft_consensus.cc:504] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.595192 2946 leader_election.cc:288] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33e5fef2a9b9494090fd50ecda44d89b (127.18.186.130:34637), 407c2eadf3d64a2e9f4406a0727b8d6c (127.18.186.131:32965)
I20260808 23:43:04.602569 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.595696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:55371 (local address 127.18.186.130:34637)
0808 23:43:04.595910 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.595918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.596519 (+ 601us) server_negotiation.cc:407] Connection header received
0808 23:43:04.596725 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.596733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.596825 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.596923 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:04.597682 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.598639 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.600689 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.601557 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.601673 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.601723 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.602142 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.602283 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:43:04.602655 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.595661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:34637 (local address 127.18.186.129:55371)
0808 23:43:04.596362 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:04.596388 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:04.596564 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:04.597045 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:04.597052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:04.597083 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:04.597524 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.597541 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.598778 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.598784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.600523 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.600536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.601778 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.601783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.601965 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.602001 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:43:04.602098 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 23:43:04.602190 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"spinlock_wait_cycles":34048,"thread_start_us":385,"threads_started":1}
I20260808 23:43:04.602722 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.596787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:50197 (local address 127.18.186.131:32965)
0808 23:43:04.596996 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.597014 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.597241 (+ 227us) server_negotiation.cc:407] Connection header received
0808 23:43:04.597397 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.597403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.597487 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.597598 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:04.598210 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.599124 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.600957 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.601783 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.601939 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.602019 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.602335 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.602466 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 23:43:04.603081 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.596396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:32965 (local address 127.18.186.129:50197)
0808 23:43:04.597141 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:04.597154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:04.597265 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:04.597636 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:04.597640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:04.597664 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:04.598050 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.598062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.599261 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.599265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.600808 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:04.600822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.601995 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.601999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:04.602162 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.602193 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:43:04.602281 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:43:04.602402 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":427,"threads_started":1}
I20260808 23:43:04.603727 2728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b01756d488a431da7ef1833ae011e77" candidate_uuid: "103435e24f5542319eae7ee1bf435ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33e5fef2a9b9494090fd50ecda44d89b" is_pre_election: true
I20260808 23:43:04.604328 2728 raft_consensus.cc:2454] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103435e24f5542319eae7ee1bf435ac2 in term 0.
I20260808 23:43:04.604230 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b01756d488a431da7ef1833ae011e77" candidate_uuid: "103435e24f5542319eae7ee1bf435ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" is_pre_election: true
I20260808 23:43:04.604846 2869 raft_consensus.cc:2454] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 103435e24f5542319eae7ee1bf435ac2 in term 0.
I20260808 23:43:04.605129 2515 leader_election.cc:302] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103435e24f5542319eae7ee1bf435ac2, 33e5fef2a9b9494090fd50ecda44d89b; no voters:
I20260808 23:43:04.605568 2946 raft_consensus.cc:2781] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:43:04.605715 2946 raft_consensus.cc:482] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:43:04.605842 2946 raft_consensus.cc:3037] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:04.608150 2946 raft_consensus.cc:504] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.608682 2946 leader_election.cc:288] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [CANDIDATE]: Term 1 election: Requested vote from peers 33e5fef2a9b9494090fd50ecda44d89b (127.18.186.130:34637), 407c2eadf3d64a2e9f4406a0727b8d6c (127.18.186.131:32965)
I20260808 23:43:04.609169 2728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b01756d488a431da7ef1833ae011e77" candidate_uuid: "103435e24f5542319eae7ee1bf435ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33e5fef2a9b9494090fd50ecda44d89b"
I20260808 23:43:04.609256 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b01756d488a431da7ef1833ae011e77" candidate_uuid: "103435e24f5542319eae7ee1bf435ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c"
I20260808 23:43:04.609380 2728 raft_consensus.cc:3037] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:04.609427 2869 raft_consensus.cc:3037] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:04.612694 2728 raft_consensus.cc:2454] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103435e24f5542319eae7ee1bf435ac2 in term 1.
I20260808 23:43:04.612694 2869 raft_consensus.cc:2454] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 103435e24f5542319eae7ee1bf435ac2 in term 1.
I20260808 23:43:04.613142 2515 leader_election.cc:302] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 103435e24f5542319eae7ee1bf435ac2, 407c2eadf3d64a2e9f4406a0727b8d6c; no voters:
I20260808 23:43:04.613405 2946 raft_consensus.cc:2781] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:43:04.613898 2946 raft_consensus.cc:686] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 1 LEADER]: Becoming Leader. State: Replica: 103435e24f5542319eae7ee1bf435ac2, State: Running, Role: LEADER
I20260808 23:43:04.614449 2946 consensus_queue.cc:237] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } }
I20260808 23:43:04.619478 2433 catalog_manager.cc:5495] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 103435e24f5542319eae7ee1bf435ac2 (127.18.186.129). New cstate: current_term: 1 leader_uuid: "103435e24f5542319eae7ee1bf435ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "103435e24f5542319eae7ee1bf435ac2" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 42715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 } health_report { overall_health: UNKNOWN } } }
W20260808 23:43:04.650295 2924 tablet.cc:2369] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:43:04.652612 2776 tablet.cc:2369] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:43:04.666527 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.661594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.18.186.129:42715)
0808 23:43:04.661790 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.661795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.661812 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:43:04.661900 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.661903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.661978 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.662103 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:43:04.662421 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.663175 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.664655 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.665078 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.665170 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.665589 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:43:04.666063 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:43:04.666180 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.666246 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.666310 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:43:04.674865 2869 raft_consensus.cc:1261] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 FOLLOWER]: Refusing update from remote peer 103435e24f5542319eae7ee1bf435ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:43:04.674901 2728 raft_consensus.cc:1261] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 1 FOLLOWER]: Refusing update from remote peer 103435e24f5542319eae7ee1bf435ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:43:04.675853 2946 consensus_queue.cc:1035] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33e5fef2a9b9494090fd50ecda44d89b" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 34637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:43:04.676178 2953 consensus_queue.cc:1035] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "407c2eadf3d64a2e9f4406a0727b8d6c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 32965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:43:04.687059 2956 mvcc.cc:205] Tried to move back new op lower bound from 7316408666817921024 to 7316408666585096192. Current Snapshot: MvccSnapshot[applied={T|T < 7316408666817921024}]
I20260808 23:43:04.687644 2957 mvcc.cc:205] Tried to move back new op lower bound from 7316408666817921024 to 7316408666585096192. Current Snapshot: MvccSnapshot[applied={T|T < 7316408666817921024}]
I20260808 23:43:04.701910 2435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:43:04.702185 2435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:43:04.705760 2435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Sending DeleteTablet for 3 replicas of tablet 5b01756d488a431da7ef1833ae011e77
I20260808 23:43:04.706985 2562 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b01756d488a431da7ef1833ae011e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:04 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:43094
I20260808 23:43:04.707129 2708 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b01756d488a431da7ef1833ae011e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:04 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:54782
I20260808 23:43:04.707792 2847 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b01756d488a431da7ef1833ae011e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:04 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53494
I20260808 23:43:04.708072 2967 tablet_replica.cc:330] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: stopping tablet replica
I20260808 23:43:04.708585 2967 raft_consensus.cc:2229] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:43:04.709230 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 2475
I20260808 23:43:04.709393 2967 raft_consensus.cc:2258] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:04.711800 2967 ts_tablet_manager.cc:1905] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:04.720522 2967 ts_tablet_manager.cc:1918] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:04.720788 2967 log.cc:1196] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/5b01756d488a431da7ef1833ae011e77
I20260808 23:43:04.721567 2967 ts_tablet_manager.cc:1939] T 5b01756d488a431da7ef1833ae011e77 P 103435e24f5542319eae7ee1bf435ac2: Deleting consensus metadata
I20260808 23:43:04.724277 2416 catalog_manager.cc:4841] TS 103435e24f5542319eae7ee1bf435ac2 (127.18.186.129:42715): tablet 5b01756d488a431da7ef1833ae011e77 (table test-table [id=28a85acce8f04c2e97f83a4329f27227]) successfully deleted
I20260808 23:43:04.727985 2969 tablet_replica.cc:330] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: stopping tablet replica
I20260808 23:43:04.728507 2969 raft_consensus.cc:2229] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:43:04.728988 2968 tablet_replica.cc:330] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: stopping tablet replica
I20260808 23:43:04.729058 2969 raft_consensus.cc:2258] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:04.730158 2968 raft_consensus.cc:2229] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:43:04.730702 2968 raft_consensus.cc:2258] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:04.730902 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.710548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43118 (local address 127.18.186.129:42715)
0808 23:43:04.710754 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.710762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.710785 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:43:04.710887 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.710895 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.711001 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.711129 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:04.712934 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.714453 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.716540 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.717550 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.717723 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.721325 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:04.721344 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:04.721366 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:04.721431 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:04.726932 (+ 5501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.727979 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.727991 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.727995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.728205 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.728680 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.728686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.728689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.729142 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:04.729471 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.730290 (+ 819us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.730535 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:43:04.731568 2626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:04.732183 2969 ts_tablet_manager.cc:1905] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:04.732967 2968 ts_tablet_manager.cc:1905] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:04.739344 2969 ts_tablet_manager.cc:1918] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:04.739550 2969 log.cc:1196] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/5b01756d488a431da7ef1833ae011e77
I20260808 23:43:04.739961 2968 ts_tablet_manager.cc:1918] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:04.740144 2968 log.cc:1196] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/5b01756d488a431da7ef1833ae011e77
I20260808 23:43:04.740177 2969 ts_tablet_manager.cc:1939] T 5b01756d488a431da7ef1833ae011e77 P 407c2eadf3d64a2e9f4406a0727b8d6c: Deleting consensus metadata
I20260808 23:43:04.740703 2968 ts_tablet_manager.cc:1939] T 5b01756d488a431da7ef1833ae011e77 P 33e5fef2a9b9494090fd50ecda44d89b: Deleting consensus metadata
I20260808 23:43:04.742202 2419 catalog_manager.cc:4841] TS 407c2eadf3d64a2e9f4406a0727b8d6c (127.18.186.131:32965): tablet 5b01756d488a431da7ef1833ae011e77 (table test-table [id=28a85acce8f04c2e97f83a4329f27227]) successfully deleted
I20260808 23:43:04.742522 2419 catalog_manager.cc:4841] TS 33e5fef2a9b9494090fd50ecda44d89b (127.18.186.130:34637): tablet 5b01756d488a431da7ef1833ae011e77 (table test-table [id=28a85acce8f04c2e97f83a4329f27227]) successfully deleted
I20260808 23:43:04.827709 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 2475
I20260808 23:43:04.846621 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 2635
I20260808 23:43:04.860657 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.847562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.18.186.130:34637)
0808 23:43:04.847809 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.847817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.847855 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:43:04.848092 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.848100 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.848212 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.848348 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:04.849658 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.850416 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.851883 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.852259 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.852358 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.855136 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:04.855150 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:04.855164 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:04.855198 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:04.857998 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.858641 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.858649 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.858651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.858772 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.859118 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.859121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.859123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.859547 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:04.859760 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.860112 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.860275 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 23:43:04.861462 2770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:04.961305 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 2635
I20260808 23:43:04.981276 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 2779
I20260808 23:43:04.995250 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:04.982099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.18.186.131:32965)
0808 23:43:04.982304 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:04.982310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:04.982331 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:43:04.982509 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:04.982513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:04.982595 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:04.982703 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:04.984189 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.984929 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.986258 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:04.986675 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:04.986811 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:04.989490 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:04.989504 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:04.989519 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:04.989550 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:04.992450 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.993138 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.993146 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.993148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.993294 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:04.993686 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:04.993692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:04.993694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:04.994051 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:04.994348 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:04.994702 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 23:43:04.994928 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:43:04.996018 2916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:05.085938 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 2779
I20260808 23:43:05.104970 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 2397
I20260808 23:43:05.106225 2467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:05.219002 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 2397
I20260808 23:43:05.245682 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:45339
--webserver_interface=127.18.186.190
--webserver_port=33637
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:45339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:05.582717 2975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:05.583034 2975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:05.583122 2975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:05.590381 2975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:43:05.590497 2975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:05.590543 2975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:05.590603 2975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:43:05.590639 2975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:43:05.600286 2975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:45339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:45339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=33637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:05.601500 2975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:05.602742 2975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:05.614547 2981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:05.614540 2982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:05.615061 2984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:05.615477 2975 server_base.cc:1034] running on GCE node
I20260808 23:43:05.616214 2975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:05.617535 2975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:05.618775 2975 hybrid_clock.cc:648] HybridClock initialized: now 1786232585618734 us; error 50 us; skew 500 ppm
I20260808 23:43:05.619161 2975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:05 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232585, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:05.622346 2975 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:43:05.623708 2975 webserver.cc:460] Webserver started at http://127.18.186.190:33637/ using document root <none> and password file <none>
I20260808 23:43:05.624298 2975 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:05.624403 2975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:05.629567 2975 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 23:43:05.632396 2993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:05.633818 2975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:43:05.633975 2975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "cec7f2500dd147f8be1c52680e0bff45"
format_stamp: "Formatted at 2026-08-08 23:42:59 on dist-test-slave-5k54"
server_key: "2c7aa51a13e5e3714441f98eb6dd96b5"
server_key_iv: "1ffe36b418bed15ed21d4e09a3f9181f"
server_key_version: "encryptionkey@0"
I20260808 23:43:05.634897 2975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:05.656728 2975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:05.660948 2975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:05.661180 2975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:05.680140 2975 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:45339
I20260808 23:43:05.680164 3050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:45339 every 8 connection(s)
I20260808 23:43:05.681716 2975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:43:05.688668 3051 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:43:05.691774 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 2975
I20260808 23:43:05.692307 3051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Bootstrap starting.
I20260808 23:43:05.692682 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:42715
--local_ip_for_outbound_sockets=127.18.186.129
--tserver_master_addrs=127.18.186.190:45339
--webserver_port=35729
--webserver_interface=127.18.186.129
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:43:05.716236 3051 log.cc:824] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:05.732493 3051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:43:05.733033 3051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45: Bootstrap complete.
I20260808 23:43:05.742956 3051 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:43:05.743980 3051 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec7f2500dd147f8be1c52680e0bff45, State: Initialized, Role: FOLLOWER
I20260808 23:43:05.744691 3051 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:43:05.744961 3051 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:43:05.745097 3051 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:43:05.745282 3051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 2 FOLLOWER]: Advancing to term 3
I20260808 23:43:05.749070 3051 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:43:05.749577 3051 leader_election.cc:302] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec7f2500dd147f8be1c52680e0bff45; no voters:
I20260808 23:43:05.750280 3051 leader_election.cc:288] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 23:43:05.750433 3056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 23:43:05.751904 3056 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [term 3 LEADER]: Becoming Leader. State: Replica: cec7f2500dd147f8be1c52680e0bff45, State: Running, Role: LEADER
I20260808 23:43:05.752678 3051 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:43:05.752544 3056 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } }
I20260808 23:43:05.755584 3059 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec7f2500dd147f8be1c52680e0bff45. Latest consensus state: current_term: 3 leader_uuid: "cec7f2500dd147f8be1c52680e0bff45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } } }
I20260808 23:43:05.756033 3059 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: This master's current role is: LEADER
I20260808 23:43:05.756603 3057 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cec7f2500dd147f8be1c52680e0bff45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7f2500dd147f8be1c52680e0bff45" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 45339 } } }
I20260808 23:43:05.756943 3057 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7f2500dd147f8be1c52680e0bff45 [sys.catalog]: This master's current role is: LEADER
I20260808 23:43:05.759126 3063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:43:05.770596 3063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:43:05.772576 3063 catalog_manager.cc:1259] Loaded cluster ID: cc4c5c74839f49ff90efd28c75149bd9
I20260808 23:43:05.772771 3063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 23:43:05.775806 3063 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8678b189ad google::LogMessage::Fail() at ??:0
@ 0x7f8678b1b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8678b184de google::LogMessage::Flush() at ??:0
@ 0x7f8678b19a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f868e54b512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f868e548976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f868e5d0610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f868e5d0439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260808 23:43:05.827945 3076 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f867b764345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f867b76791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f867b767799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f867b742eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f867ea1c6db start_thread at ??:0
@ 0x7f86779b261f clone at ??:0
W20260808 23:43:06.079674 3053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:06.080003 3053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:06.080174 3053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:06.087694 3053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:06.087839 3053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:06.088044 3053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:43:06.097800 3053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:42715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=35729
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:06.099107 3053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:06.100518 3053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:06.113867 3097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:06.113970 3098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:06.113951 3100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:06.114838 3053 server_base.cc:1034] running on GCE node
I20260808 23:43:06.115417 3053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:06.116403 3053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:06.117647 3053 hybrid_clock.cc:648] HybridClock initialized: now 1786232586117606 us; error 36 us; skew 500 ppm
I20260808 23:43:06.117960 3053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:06 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232586, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:06.121197 3053 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:43:06.122630 3053 webserver.cc:460] Webserver started at http://127.18.186.129:35729/ using document root <none> and password file <none>
I20260808 23:43:06.123359 3053 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:06.123564 3053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:06.128729 3053 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:43:06.131373 3107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:06.132753 3053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:43:06.132936 3053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "103435e24f5542319eae7ee1bf435ac2"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "f1f5e0fdd370e956a9f102df5048f24b"
server_key_iv: "a3dd72fdff8479fdc6c6fa66fd95c495"
server_key_version: "encryptionkey@0"
I20260808 23:43:06.133901 3053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:06.149055 3053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:06.153271 3053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:06.153589 3053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:06.154542 3053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:06.155900 3053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:06.156032 3053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:06.156165 3053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:06.156265 3053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:06.191982 3053 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:42715
I20260808 23:43:06.192075 3223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:42715 every 8 connection(s)
I20260808 23:43:06.193475 3053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:43:06.195570 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 3053
I20260808 23:43:06.196210 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:34637
--local_ip_for_outbound_sockets=127.18.186.130
--tserver_master_addrs=127.18.186.190:45339
--webserver_port=38099
--webserver_interface=127.18.186.130
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:43:06.198799 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:06.196762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:53109)
0808 23:43:06.197601 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:06.197847 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":362,"threads_started":1}
W20260808 23:43:06.199561 3224 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:45339 (0 consecutive failures): Network error: Failed to ping master at 127.18.186.190:45339: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
I20260808 23:43:06.200752 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:06.200256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:55295)
0808 23:43:06.200481 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:06.200583 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 23:43:06.201797 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:06.201339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:60261)
0808 23:43:06.201514 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:06.201624 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 23:43:06.202061 3224 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:43:06.532449 3227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:06.532732 3227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:06.532935 3227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:06.540491 3227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:06.540621 3227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:06.540799 3227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:43:06.550346 3227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:34637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=38099
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:06.551702 3227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:06.553012 3227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:06.566421 3232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:06.566464 3233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:06.566366 3235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:06.567171 3227 server_base.cc:1034] running on GCE node
I20260808 23:43:06.567817 3227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:06.568763 3227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:06.572054 3227 hybrid_clock.cc:648] HybridClock initialized: now 1786232586572011 us; error 33 us; skew 500 ppm
I20260808 23:43:06.572377 3227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:06 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232586, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:06.575536 3227 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:43:06.576939 3227 webserver.cc:460] Webserver started at http://127.18.186.130:38099/ using document root <none> and password file <none>
I20260808 23:43:06.577633 3227 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:06.577767 3227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:06.582968 3227 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:43:06.585655 3243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:06.586995 3227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:43:06.587177 3227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "33e5fef2a9b9494090fd50ecda44d89b"
format_stamp: "Formatted at 2026-08-08 23:43:00 on dist-test-slave-5k54"
server_key: "cfa1dfc256102919fde278e04a067acf"
server_key_iv: "e31a3beb9ebf65c1764be68358cccdf5"
server_key_version: "encryptionkey@0"
I20260808 23:43:06.588251 3227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:06.606101 3227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:06.609879 3227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:06.610136 3227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:06.611089 3227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:06.612483 3227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:06.612596 3227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:06.612726 3227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:06.612818 3227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:06.647471 3227 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:34637
I20260808 23:43:06.647571 3368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:34637 every 8 connection(s)
I20260808 23:43:06.648995 3227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:43:06.654170 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 3227
I20260808 23:43:06.654304 3372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:06.652431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:59219)
0808 23:43:06.653175 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:06.653389 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":287,"threads_started":1}
W20260808 23:43:06.654979 3369 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:45339 (0 consecutive failures): Network error: Failed to ping master at 127.18.186.190:45339: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
I20260808 23:43:06.655019 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:32965
--local_ip_for_outbound_sockets=127.18.186.131
--tserver_master_addrs=127.18.186.190:45339
--webserver_port=39093
--webserver_interface=127.18.186.131
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:43:06.655920 3372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:06.655603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:34573)
0808 23:43:06.655743 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:06.655807 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:43:06.656787 3372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:06.656501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:50887)
0808 23:43:06.656623 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:06.656690 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 23:43:06.657076 3369 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:43:06.986009 3373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:06.986275 3373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:06.986450 3373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:06.993661 3373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:06.993781 3373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:06.993963 3373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:43:07.003386 3373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:32965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=39093
--tserver_master_addrs=127.18.186.190:45339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:07.004674 3373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:07.005987 3373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:07.019536 3379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:07.019549 3381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:07.019809 3373 server_base.cc:1034] running on GCE node
W20260808 23:43:07.019577 3378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:07.020686 3373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:07.021608 3373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:07.022805 3373 hybrid_clock.cc:648] HybridClock initialized: now 1786232587022750 us; error 51 us; skew 500 ppm
I20260808 23:43:07.023134 3373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:07 dist-test-slave-5k54 krb5kdc[1801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232587, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:07.026427 3373 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:43:07.027961 3373 webserver.cc:460] Webserver started at http://127.18.186.131:39093/ using document root <none> and password file <none>
I20260808 23:43:07.028641 3373 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:07.028785 3373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:07.033954 3373 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:43:07.036808 3389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:07.038220 3373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:43:07.038393 3373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "407c2eadf3d64a2e9f4406a0727b8d6c"
format_stamp: "Formatted at 2026-08-08 23:43:01 on dist-test-slave-5k54"
server_key: "fa04d879ecba39fdbbf3300f71a2e267"
server_key_iv: "36e32721262cf1ff17093f9ed5f96a78"
server_key_version: "encryptionkey@0"
I20260808 23:43:07.039409 3373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:07.061591 3373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:07.065539 3373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:07.065807 3373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:07.066761 3373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:07.068099 3373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:07.068221 3373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:07.068352 3373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:07.068445 3373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:07.102278 3373 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:32965
I20260808 23:43:07.102383 3509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:32965 every 8 connection(s)
I20260808 23:43:07.103906 3373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:43:07.108690 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:07.106842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:51009)
0808 23:43:07.107619 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:07.107854 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":275,"threads_started":1}
W20260808 23:43:07.109582 3510 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:45339 (0 consecutive failures): Network error: Failed to ping master at 127.18.186.190:45339: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
I20260808 23:43:07.110424 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:07.110087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:36507)
0808 23:43:07.110221 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:07.110318 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 23:43:07.111402 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:07.110982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:53357)
0808 23:43:07.111209 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:07.111288 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260808 23:43:07.111769 3510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:43:07.112924 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 3373
I20260808 23:43:07.203686 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:07.202692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:52275)
0808 23:43:07.203283 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:07.203412 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":355,"threads_started":1}
I20260808 23:43:07.658495 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:07.657621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:52989)
0808 23:43:07.658121 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:07.658250 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":269,"threads_started":1}
I20260808 23:43:08.113013 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:08.112238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:41413)
0808 23:43:08.112674 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:08.112791 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":208,"threads_started":1}
I20260808 23:43:08.205374 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:08.204625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:60195)
0808 23:43:08.205139 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:08.205216 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":264,"threads_started":1}
I20260808 23:43:08.660282 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:08.659395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:43571)
0808 23:43:08.660011 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:08.660100 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":291,"threads_started":1}
I20260808 23:43:09.114670 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:09.113945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:48235)
0808 23:43:09.114426 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:09.114506 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":249,"threads_started":1}
I20260808 23:43:09.206816 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:09.206154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:43223)
0808 23:43:09.206579 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:09.206657 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":226,"threads_started":1}
I20260808 23:43:09.661909 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:09.661192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:50479)
0808 23:43:09.661655 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:09.661735 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":261,"threads_started":1}
I20260808 23:43:10.116366 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:10.115609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:55845)
0808 23:43:10.116109 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:10.116190 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":301,"threads_started":1}
I20260808 23:43:10.208469 3526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:10.207757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:50073)
0808 23:43:10.208211 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:10.208288 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":252,"threads_started":1}
I20260808 23:43:10.663669 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:10.662760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:47389)
0808 23:43:10.663315 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:10.663397 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":333,"threads_started":1}
I20260808 23:43:11.118108 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:11.117262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:33469)
0808 23:43:11.117813 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:11.117899 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":323,"threads_started":1}
I20260808 23:43:11.210062 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:11.209337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:60261)
0808 23:43:11.209809 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:11.209893 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":218,"threads_started":1}
I20260808 23:43:11.665287 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:11.664516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:49415)
0808 23:43:11.665013 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:11.665104 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":259,"threads_started":1}
I20260808 23:43:12.119885 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:12.119091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:50309)
0808 23:43:12.119644 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:12.119720 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":258,"threads_started":1}
I20260808 23:43:12.211714 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:12.210971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:40837)
0808 23:43:12.211396 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:12.211556 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":225,"threads_started":1}
I20260808 23:43:12.666941 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:12.666199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:58181)
0808 23:43:12.666668 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:12.666762 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":250,"threads_started":1}
I20260808 23:43:13.121542 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:13.120763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:34959)
0808 23:43:13.121290 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:13.121381 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":277,"threads_started":1}
I20260808 23:43:13.213476 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:13.212619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:46483)
0808 23:43:13.213136 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:13.213252 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":241,"threads_started":1}
I20260808 23:43:13.668857 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:13.668062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:59619)
0808 23:43:13.668554 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:13.668637 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":214,"threads_started":1}
I20260808 23:43:14.123634 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:14.122720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:57731)
0808 23:43:14.123248 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:14.123338 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":238,"threads_started":1}
I20260808 23:43:14.215126 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:14.214411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:60603)
0808 23:43:14.214860 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:14.214958 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":241,"threads_started":1}
I20260808 23:43:14.670481 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:14.669760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:41665)
0808 23:43:14.670217 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:14.670302 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":248,"threads_started":1}
I20260808 23:43:15.125335 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:15.124625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:38947)
0808 23:43:15.125084 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:15.125174 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":249,"threads_started":1}
I20260808 23:43:15.216750 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:15.216009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:40983)
0808 23:43:15.216495 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:15.216578 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":253,"threads_started":1}
I20260808 23:43:15.672439 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:15.671579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:38825)
0808 23:43:15.672179 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:15.672260 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":329,"threads_started":1}
I20260808 23:43:16.126993 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:16.126241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:56679)
0808 23:43:16.126737 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:16.126820 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":256,"threads_started":1}
I20260808 23:43:16.218294 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:16.217636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:46125)
0808 23:43:16.218058 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:16.218139 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":229,"threads_started":1}
I20260808 23:43:16.674176 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:16.673491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:33373)
0808 23:43:16.673925 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:16.674015 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":252,"threads_started":1}
I20260808 23:43:17.128708 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:17.127988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:43805)
0808 23:43:17.128461 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:17.128542 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":260,"threads_started":1}
I20260808 23:43:17.219975 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:17.219219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:40491)
0808 23:43:17.219714 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:17.219803 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":296,"threads_started":1}
I20260808 23:43:17.675925 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:17.675160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:41165)
0808 23:43:17.675653 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:17.675744 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":279,"threads_started":1}
I20260808 23:43:18.130501 3550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:18.129671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:34357)
0808 23:43:18.130211 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:18.130307 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":331,"threads_started":1}
I20260808 23:43:18.221619 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:18.220912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:52453)
0808 23:43:18.221376 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:18.221460 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":229,"threads_started":1}
I20260808 23:43:18.677543 3552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:18.676810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:50889)
0808 23:43:18.677305 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:18.677384 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":282,"threads_started":1}
I20260808 23:43:19.132234 3553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:19.131573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:54789)
0808 23:43:19.132000 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:19.132080 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":208,"threads_started":1}
I20260808 23:43:19.223409 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:19.222525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:45629)
0808 23:43:19.223052 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:19.223169 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":222,"threads_started":1}
I20260808 23:43:19.679281 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:19.678491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:40305)
0808 23:43:19.678998 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:19.679093 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":250,"threads_started":1}
I20260808 23:43:20.134002 3556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:20.133150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:47913)
0808 23:43:20.133720 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:20.133812 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":328,"threads_started":1}
I20260808 23:43:20.225014 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:20.224273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:55057)
0808 23:43:20.224725 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:20.224819 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":247,"threads_started":1}
I20260808 23:43:20.681044 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:20.680297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:60317)
0808 23:43:20.680755 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:20.680831 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":254,"threads_started":1}
I20260808 23:43:21.135802 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:21.135022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:39311)
0808 23:43:21.135534 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:21.135618 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":244,"threads_started":1}
I20260808 23:43:21.226869 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:21.226101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.129:42959)
0808 23:43:21.226567 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:21.226667 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":246,"threads_started":1}
I20260808 23:43:21.682865 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:21.682143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.130:50107)
0808 23:43:21.682585 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:21.682680 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260808 23:43:22.116986 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 3053
I20260808 23:43:22.134511 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:22.118071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.18.186.129:42715)
0808 23:43:22.118805 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:22.118821 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:22.118862 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:43:22.120603 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:22.120613 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:22.121051 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:22.121362 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:22.122739 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:22.124013 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:22.125241 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:22.125877 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:22.126012 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:22.128676 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:22.128684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:22.128698 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:22.128736 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:22.131711 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:22.132310 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:22.132316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:22.132321 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:22.132500 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:22.132835 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:22.132838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:22.132840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:22.133328 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:22.133560 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:22.133919 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 23:43:22.134228 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":369,"threads_started":1}
I20260808 23:43:22.135965 3218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:22.137915 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:22.136921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:45339 (local address 127.18.186.131:59803)
0808 23:43:22.137513 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:22.137684 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:45339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":331,"threads_started":1}
I20260808 23:43:22.221951 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 3053
I20260808 23:43:22.240350 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 3227
I20260808 23:43:22.258216 3565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:22.241419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.18.186.130:34637)
0808 23:43:22.242051 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:22.242064 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:22.242102 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:43:22.243995 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:22.244007 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:22.244483 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:22.244780 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:22.246136 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:22.247363 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:22.248661 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:22.249322 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:22.249453 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:22.252224 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:22.252231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:22.252244 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:22.252284 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:22.255335 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:22.256023 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:22.256030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:22.256035 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:22.256192 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:22.256501 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:22.256504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:22.256505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:22.256995 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:22.257217 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:22.257520 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 23:43:22.257935 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":317,"threads_started":1}
I20260808 23:43:22.259605 3364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:22.345705 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 3227
I20260808 23:43:22.366698 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 3373
I20260808 23:43:22.383937 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:22.367672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.18.186.131:32965)
0808 23:43:22.368291 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:22.368305 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:22.368356 (+ 51us) server_negotiation.cc:407] Connection header received
0808 23:43:22.370023 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:22.370032 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:22.370472 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:22.370759 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:22.372212 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:22.373436 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:22.374700 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:22.375340 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:22.375549 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:22.378022 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:22.378029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:22.378043 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:22.378078 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:22.380984 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:22.381604 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:22.381610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:22.381616 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:22.381775 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:22.382125 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:22.382128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:22.382130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:22.382657 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:22.382889 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:22.383261 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 23:43:22.383644 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":306,"threads_started":1}
I20260808 23:43:22.385315 3505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:22.469416 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 3373
2026-08-08T23:43:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24423 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:43:22 dist-test-slave-5k54 krb5kdc[3571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:43:22 dist-test-slave-5k54 krb5kdc[3571](info): set up 2 sockets
Aug 08 23:43:22 dist-test-slave-5k54 krb5kdc[3571](info): commencing operation
krb5kdc: starting...
W20260808 23:43:23.639003 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:43:23 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:43:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:43:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:43:23.911917 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42427
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:24.252221 3585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:24.252563 3585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:24.252735 3585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:24.260143 3585 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:43:24.260267 3585 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:43:24.260339 3585 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:43:24.260421 3585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:43:24.260499 3585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:24.260604 3585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:24.260696 3585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:43:24.260778 3585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:43:24.270961 3585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:24.272383 3585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:24.273769 3585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:24.285806 3593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:24.285881 3594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:24.285856 3596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:24.286731 3585 server_base.cc:1034] running on GCE node
I20260808 23:43:24.287482 3585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:24.288807 3585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:24.290066 3585 hybrid_clock.cc:648] HybridClock initialized: now 1786232604289990 us; error 49 us; skew 500 ppm
I20260808 23:43:24.290449 3585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:24 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:24.293684 3585 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:43:24.295046 3585 webserver.cc:460] Webserver started at http://127.18.186.190:43799/ using document root <none> and password file <none>
I20260808 23:43:24.295776 3585 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:24.295864 3585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:24.296129 3585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:43:24.299000 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "36a7abc22b4f458b8504f82c27c07375"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "1a24c49f70df55cf0eca9aada240e2b9"
server_key_iv: "69298c773cd1ade87c61577f093f5ee7"
server_key_version: "encryptionkey@0"
I20260808 23:43:24.299793 3585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "36a7abc22b4f458b8504f82c27c07375"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "1a24c49f70df55cf0eca9aada240e2b9"
server_key_iv: "69298c773cd1ade87c61577f093f5ee7"
server_key_version: "encryptionkey@0"
I20260808 23:43:24.304785 3585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 23:43:24.309299 3603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:24.310736 3585 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:43:24.310877 3585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "36a7abc22b4f458b8504f82c27c07375"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "1a24c49f70df55cf0eca9aada240e2b9"
server_key_iv: "69298c773cd1ade87c61577f093f5ee7"
server_key_version: "encryptionkey@0"
I20260808 23:43:24.311064 3585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:24.350741 3585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:24.354707 3585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:24.354929 3585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:24.373857 3585 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42427
I20260808 23:43:24.373868 3660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42427 every 8 connection(s)
I20260808 23:43:24.375212 3585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:43:24.378221 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 3585
I20260808 23:43:24.378468 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:43:24.379042 3661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:24.379180 19178 external_mini_cluster.cc:1428] Setting key 300eeeb55af57fe524e0b087886ac893
I20260808 23:43:24.393471 3661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Bootstrap starting.
Aug 08 23:43:24 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:24.397110 3661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:24.398502 3661 log.cc:824] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:24.401866 3661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: No bootstrap required, opened a new log
I20260808 23:43:24.404045 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:24.381155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52444 (local address 127.18.186.190:42427)
0808 23:43:24.381972 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:24.381983 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:24.382024 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:43:24.383716 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:24.383738 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:24.384148 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:24.384462 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:24.386306 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:24.387636 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:24.389671 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:24.390382 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:24.390523 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:24.395404 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:24.395415 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:24.395454 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:24.395505 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:24.399520 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:24.400293 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:24.400301 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:24.400308 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:24.400521 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:24.400879 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:24.400883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:24.400886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:24.401584 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:24.401872 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:24.402248 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 23:43:24.402717 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":375,"threads_started":1}
I20260808 23:43:24.410496 3661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:43:24.410892 3661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:24.411033 3661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a7abc22b4f458b8504f82c27c07375, State: Initialized, Role: FOLLOWER
I20260808 23:43:24.411618 3661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:43:24.411868 3661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:43:24.412016 3661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:43:24.412185 3661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:24.414376 3661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:43:24.414845 3661 leader_election.cc:302] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a7abc22b4f458b8504f82c27c07375; no voters:
I20260808 23:43:24.415511 3661 leader_election.cc:288] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:43:24.415639 3666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:43:24.416384 3666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 1 LEADER]: Becoming Leader. State: Replica: 36a7abc22b4f458b8504f82c27c07375, State: Running, Role: LEADER
I20260808 23:43:24.416925 3666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:43:24.417491 3661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:43:24.419915 3668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a7abc22b4f458b8504f82c27c07375. Latest consensus state: current_term: 1 leader_uuid: "36a7abc22b4f458b8504f82c27c07375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } } }
I20260808 23:43:24.420318 3668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: This master's current role is: LEADER
I20260808 23:43:24.420768 3667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a7abc22b4f458b8504f82c27c07375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } } }
I20260808 23:43:24.421100 3667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: This master's current role is: LEADER
I20260808 23:43:24.422262 3674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:43:24.428285 3674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:43:24.436995 3674 catalog_manager.cc:1347] Generated new cluster ID: bbe6cd9a9b86495abee67736a9ddab7b
I20260808 23:43:24.437110 3674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:43:24.453320 3674 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:43:24.455163 3674 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:43:24.471800 3674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Generated new TSK 0
I20260808 23:43:24.472911 3674 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:43:24.546710 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42427
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:24.888015 3687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:24.888280 3687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:24.888374 3687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:24.895624 3687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:24.895753 3687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:24.895905 3687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:43:24.905721 3687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:24.907017 3687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:24.908408 3687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:24.921908 3692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:24.921878 3695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:24.921965 3693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:24.925146 3687 server_base.cc:1034] running on GCE node
I20260808 23:43:24.925788 3687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:24.926781 3687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:24.928002 3687 hybrid_clock.cc:648] HybridClock initialized: now 1786232604927914 us; error 79 us; skew 500 ppm
I20260808 23:43:24.928306 3687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:24 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:24.931514 3687 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:43:24.932893 3687 webserver.cc:460] Webserver started at http://127.18.186.129:33801/ using document root <none> and password file <none>
I20260808 23:43:24.933547 3687 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:24.933647 3687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:24.933895 3687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:43:24.936868 3687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "8c5490bf8733411f9f590e08eaf22f07"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "d81a81a3d74d446018ae7e97e4a23f45"
server_key_iv: "35f5209dbb8bb0dfdc2a5783cb702c54"
server_key_version: "encryptionkey@0"
I20260808 23:43:24.937603 3687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "8c5490bf8733411f9f590e08eaf22f07"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "d81a81a3d74d446018ae7e97e4a23f45"
server_key_iv: "35f5209dbb8bb0dfdc2a5783cb702c54"
server_key_version: "encryptionkey@0"
I20260808 23:43:24.942561 3687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:43:24.945940 3704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:24.947274 3687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:43:24.947398 3687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "8c5490bf8733411f9f590e08eaf22f07"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "d81a81a3d74d446018ae7e97e4a23f45"
server_key_iv: "35f5209dbb8bb0dfdc2a5783cb702c54"
server_key_version: "encryptionkey@0"
I20260808 23:43:24.947640 3687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:24.969480 3687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:24.973611 3687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:24.973848 3687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:24.974787 3687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:24.976073 3687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:24.976154 3687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:24.976248 3687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:24.976292 3687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:25.010557 3687 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:38707
I20260808 23:43:25.010637 3828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:38707 every 8 connection(s)
I20260808 23:43:25.011905 3687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:43:25.018612 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 3687
I20260808 23:43:25.018875 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:43:25.019629 19178 external_mini_cluster.cc:1428] Setting key f230ab89fd676e4a328454bdce88156f
Aug 08 23:43:25 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:25.034102 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:25.014542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:52759 (local address 127.18.186.190:42427)
0808 23:43:25.015076 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:25.015080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:25.016052 (+ 972us) server_negotiation.cc:407] Connection header received
0808 23:43:25.017842 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:25.017847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:25.017971 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:25.018066 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:25.019783 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:25.020567 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:25.022603 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:25.023308 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:25.023476 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:25.027278 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:25.027287 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:25.027295 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:25.027339 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:25.031016 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:25.031817 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:25.031825 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:25.031831 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:25.031994 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:25.032402 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:25.032410 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:25.032415 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:25.032786 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:25.032947 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:25.033663 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 23:43:25.033843 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":246,"threads_started":1}
I20260808 23:43:25.035753 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:25.015101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:52759)
0808 23:43:25.015873 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:25.015919 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:25.017593 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:25.018315 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:25.018324 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:25.018898 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:25.019590 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:25.019615 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:25.020771 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:25.020776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:25.022440 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:25.022452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:25.023560 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:25.023565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:25.023851 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:25.024589 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:25.024624 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:25.027037 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:25.031159 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:25.031165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:25.031176 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:25.031685 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:25.032100 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:25.032103 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:25.032106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:25.032275 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:25.032916 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:25.032922 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:25.033217 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 23:43:25.033582 (+ 365us) client_negotiation.cc:240] Negotiation successful
0808 23:43:25.034214 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":345,"threads_started":1}
I20260808 23:43:25.037292 3830 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42427
I20260808 23:43:25.037609 3830 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:25.038296 3830 heartbeater.cc:511] Master 127.18.186.190:42427 requested a full tablet report, sending...
I20260808 23:43:25.040390 3622 ts_manager.cc:194] Registered new tserver with Master: 8c5490bf8733411f9f590e08eaf22f07 (127.18.186.129:38707)
I20260808 23:43:25.041839 3622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:52759
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:43:25.083060 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42427
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:25.423676 3836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:25.423954 3836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:25.424149 3836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:25.431316 3836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:25.431440 3836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:25.431622 3836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:43:25.441080 3836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:25.442360 3836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:25.443691 3836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:25.456851 3841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:25.456815 3844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:25.456878 3842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:25.458070 3836 server_base.cc:1034] running on GCE node
I20260808 23:43:25.458666 3836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:25.459676 3836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:25.460942 3836 hybrid_clock.cc:648] HybridClock initialized: now 1786232605460847 us; error 78 us; skew 500 ppm
I20260808 23:43:25.461356 3836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:25 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232605, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:25.464929 3836 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:43:25.466550 3836 webserver.cc:460] Webserver started at http://127.18.186.130:39021/ using document root <none> and password file <none>
I20260808 23:43:25.467298 3836 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:25.467494 3836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:25.467836 3836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:43:25.471009 3836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "a66f85187830733e639782abfb301f68"
server_key_iv: "253d18886a5891a886b90ff147894090"
server_key_version: "encryptionkey@0"
I20260808 23:43:25.471920 3836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "a66f85187830733e639782abfb301f68"
server_key_iv: "253d18886a5891a886b90ff147894090"
server_key_version: "encryptionkey@0"
I20260808 23:43:25.477442 3836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260808 23:43:25.480832 3851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:25.482122 3836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:43:25.482295 3836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "a66f85187830733e639782abfb301f68"
server_key_iv: "253d18886a5891a886b90ff147894090"
server_key_version: "encryptionkey@0"
I20260808 23:43:25.482522 3836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:25.505520 3836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:25.509846 3836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:25.510130 3836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:25.511075 3836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:25.512465 3836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:25.512545 3836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:25.512627 3836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:25.512687 3836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:25.546258 3836 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:44011
I20260808 23:43:25.546337 3968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:44011 every 8 connection(s)
I20260808 23:43:25.547660 3836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:43:25.549830 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 3836
I20260808 23:43:25.550051 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:43:25.550722 19178 external_mini_cluster.cc:1428] Setting key 8c45af32521a591449bda881d11a3542
Aug 08 23:43:25 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232605, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:25.571018 3972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:25.550271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:34589 (local address 127.18.186.190:42427)
0808 23:43:25.550761 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:25.550768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:25.551906 (+ 1138us) server_negotiation.cc:407] Connection header received
0808 23:43:25.554380 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:25.554384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:25.554448 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:25.554533 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:25.556685 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:25.557383 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:25.559904 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:25.560347 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:25.560454 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:25.564945 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:25.564950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:25.564954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:25.564988 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:25.567940 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:25.568673 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:25.568677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:25.568679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:25.568805 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:25.569242 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:25.569246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:25.569249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:25.569523 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:25.569682 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:25.570364 (+ 682us) server_negotiation.cc:299] Negotiation successful
0808 23:43:25.570593 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":227,"threads_started":1}
I20260808 23:43:25.572059 3973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:25.550868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:34589)
0808 23:43:25.551746 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:25.551784 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:25.554076 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:25.554821 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:25.554837 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:25.555691 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:25.556513 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:25.556542 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:25.557554 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:25.557562 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:25.559721 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:25.559739 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:25.560584 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:25.560594 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:25.560923 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:25.561859 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:25.561893 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:25.564675 (+ 2782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:25.568078 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:25.568088 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:25.568105 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:25.568537 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:25.568917 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:25.568923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:25.568927 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:25.569117 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:25.569653 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:25.569661 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:25.570002 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 23:43:25.570266 (+ 264us) client_negotiation.cc:240] Negotiation successful
0808 23:43:25.570804 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":350,"threads_started":1}
I20260808 23:43:25.573661 3969 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42427
I20260808 23:43:25.573966 3969 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:25.574684 3969 heartbeater.cc:511] Master 127.18.186.190:42427 requested a full tablet report, sending...
I20260808 23:43:25.576225 3622 ts_manager.cc:194] Registered new tserver with Master: c51a5b5485804b7ba7c1736e0b0cd6f9 (127.18.186.130:44011)
I20260808 23:43:25.577574 3622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:34589
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:43:25.617179 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42427
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:25.948601 3976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:25.948952 3976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:25.949131 3976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:25.956267 3976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:25.956367 3976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:25.956537 3976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:43:25.966009 3976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:25.967248 3976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:25.968591 3976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:25.981709 3981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:25.981698 3986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:25.981698 3983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:25.982492 3976 server_base.cc:1034] running on GCE node
I20260808 23:43:25.983116 3976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:25.984071 3976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:25.985260 3976 hybrid_clock.cc:648] HybridClock initialized: now 1786232605985202 us; error 59 us; skew 500 ppm
I20260808 23:43:25.985625 3976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:25 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232605, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:25.988907 3976 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:43:25.990309 3976 webserver.cc:460] Webserver started at http://127.18.186.131:33565/ using document root <none> and password file <none>
I20260808 23:43:25.990938 3976 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:25.991065 3976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:25.991366 3976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:43:25.994156 3976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "09864c13a4e74fc6b3f55b82448b112c"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "1648097478eb6549e0b73305d9307ebd"
server_key_iv: "6f03ef7f05ce1881605ec4c91b2235f5"
server_key_version: "encryptionkey@0"
I20260808 23:43:25.994946 3976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "09864c13a4e74fc6b3f55b82448b112c"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "1648097478eb6549e0b73305d9307ebd"
server_key_iv: "6f03ef7f05ce1881605ec4c91b2235f5"
server_key_version: "encryptionkey@0"
I20260808 23:43:25.999974 3976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 23:43:26.003396 3993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:26.004871 3976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:43:26.005084 3976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "09864c13a4e74fc6b3f55b82448b112c"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "1648097478eb6549e0b73305d9307ebd"
server_key_iv: "6f03ef7f05ce1881605ec4c91b2235f5"
server_key_version: "encryptionkey@0"
I20260808 23:43:26.005272 3976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:26.031502 3976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:26.035404 3976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:26.035775 3976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:26.036762 3976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:26.038065 3976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:26.038141 3976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:26.038309 3976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:26.038362 3976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:26.044942 3830 heartbeater.cc:503] Master 127.18.186.190:42427 was elected leader, sending a full tablet report...
I20260808 23:43:26.073975 3976 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:36605
I20260808 23:43:26.074052 4112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:36605 every 8 connection(s)
I20260808 23:43:26.075264 3976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:43:26.083877 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 3976
I20260808 23:43:26.084123 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:43:26.084604 19178 external_mini_cluster.cc:1428] Setting key 3c62235e52c14f63ca9d192ff31a5497
Aug 08 23:43:26 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232605, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:26.095270 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.077843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:56749 (local address 127.18.186.190:42427)
0808 23:43:26.078338 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.078343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.079141 (+ 798us) server_negotiation.cc:407] Connection header received
0808 23:43:26.081065 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.081071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.081179 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.081300 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:26.082811 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.083820 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.085702 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.086357 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.086480 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.090102 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:26.090107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:26.090110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:26.090142 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:26.092662 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.093316 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.093320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.093322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.093436 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.093788 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.093791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.093793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.094074 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:26.094192 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.094802 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.094993 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":256,"threads_started":1}
I20260808 23:43:26.096271 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.078259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:56749)
0808 23:43:26.078975 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:26.079018 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:26.080783 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:26.081543 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:26.081552 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:26.082063 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:26.082634 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.082651 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.084032 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.084039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.085571 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.085584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.086635 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.086642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.086903 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.087736 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:26.087766 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:26.089920 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:26.092785 (+ 2865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:26.092791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:26.092804 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:26.093218 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:26.093527 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:26.093530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:26.093532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:26.093698 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:26.094185 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:26.094191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:26.094501 (+ 310us) client_negotiation.cc:769] Sending connection context
0808 23:43:26.094727 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 23:43:26.095115 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":289,"threads_started":1}
I20260808 23:43:26.097756 4113 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42427
I20260808 23:43:26.098060 4113 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:26.098966 4113 heartbeater.cc:511] Master 127.18.186.190:42427 requested a full tablet report, sending...
I20260808 23:43:26.100648 3622 ts_manager.cc:194] Registered new tserver with Master: 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605)
I20260808 23:43:26.101428 3622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:56749
I20260808 23:43:26.113104 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:43:26.131198 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.118059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.18.186.190:42427)
0808 23:43:26.118244 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.118249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.118788 (+ 539us) server_negotiation.cc:407] Connection header received
0808 23:43:26.118971 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.118975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.119059 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.119152 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:26.120706 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.121516 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.122927 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.123361 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.123549 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.125918 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:26.125923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:26.125927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:26.125962 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:26.128683 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.129386 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.129390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.129393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.129536 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.129907 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.129910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.129913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.130251 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:26.130453 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.130740 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.130959 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:43:26.135360 3622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:43:26.142324 3622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:43:26.166999 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.155219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57940 (local address 127.18.186.129:38707)
0808 23:43:26.159553 (+ 4334us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.159561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.159587 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:43:26.159694 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.159702 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.159906 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.160121 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:26.160652 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.161910 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.164644 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.165625 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.165847 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.166038 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.166485 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.166662 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4116,"thread_start_us":3931,"threads_started":1}
I20260808 23:43:26.166961 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.154715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:38707 (local address 127.0.0.1:57940)
0808 23:43:26.156727 (+ 2012us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:26.156784 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:26.156995 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:26.160096 (+ 3101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:26.160103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:26.160131 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:26.160492 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.160502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.162106 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.162111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.164493 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.164503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.165845 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.165861 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.166053 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.166118 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 23:43:26.166273 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 23:43:26.166478 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1768,"mutex_wait_us":128,"thread_start_us":2623,"threads_started":1}
I20260808 23:43:26.167601 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.155121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44054 (local address 127.18.186.130:44011)
0808 23:43:26.156193 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.156204 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.157850 (+ 1646us) server_negotiation.cc:407] Connection header received
0808 23:43:26.157986 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.157995 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.158302 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.158537 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:26.159553 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.161229 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.164298 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.165699 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.165870 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.166048 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.167080 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.167256 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":753,"threads_started":1}
I20260808 23:43:26.168071 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.154967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:44011 (local address 127.0.0.1:44054)
0808 23:43:26.157591 (+ 2624us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:26.157617 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:26.157724 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:26.158855 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:26.158862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:26.158892 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:26.159337 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.159349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.161384 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.161390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.164122 (+ 2732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.164137 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.166273 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.166281 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.166480 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.166512 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:43:26.166590 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 23:43:26.166727 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2509,"mutex_wait_us":3,"thread_start_us":2158,"threads_started":1}
I20260808 23:43:26.168501 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.156097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:36605 (local address 127.0.0.1:57736)
0808 23:43:26.158231 (+ 2134us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:26.158248 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:26.158358 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:26.159598 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:26.159602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:26.159634 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:26.160024 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.160037 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.161925 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.161931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.164016 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.164032 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.166236 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.166252 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.167743 (+ 1491us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.167778 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:43:26.167909 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 23:43:26.168043 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1991,"mutex_wait_us":1284,"thread_start_us":1844,"threads_started":1}
I20260808 23:43:26.168569 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.156682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.18.186.131:36605)
0808 23:43:26.158932 (+ 2250us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.158942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.158974 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:43:26.159078 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.159088 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.159359 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.159573 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:26.160260 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.161765 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.164183 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.165699 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.167111 (+ 1412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.167290 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.167949 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.168118 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2070,"thread_start_us":1923,"threads_started":1}
I20260808 23:43:26.172214 4044 tablet_service.cc:1467] Processing CreateTablet for tablet 85a9f34946894221b70d73ddb12bcc0f (DEFAULT_TABLE table=test-table [id=c4a4f7c3466d499b84b1f56108ab91d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:26.172039 3755 tablet_service.cc:1467] Processing CreateTablet for tablet 85a9f34946894221b70d73ddb12bcc0f (DEFAULT_TABLE table=test-table [id=c4a4f7c3466d499b84b1f56108ab91d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:26.172036 3901 tablet_service.cc:1467] Processing CreateTablet for tablet 85a9f34946894221b70d73ddb12bcc0f (DEFAULT_TABLE table=test-table [id=c4a4f7c3466d499b84b1f56108ab91d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:26.173437 4044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85a9f34946894221b70d73ddb12bcc0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:26.173499 3901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85a9f34946894221b70d73ddb12bcc0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:26.173501 3755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85a9f34946894221b70d73ddb12bcc0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:26.186462 4134 tablet_bootstrap.cc:492] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Bootstrap starting.
I20260808 23:43:26.188277 4135 tablet_bootstrap.cc:492] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Bootstrap starting.
I20260808 23:43:26.190900 4134 tablet_bootstrap.cc:654] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:26.191197 4135 tablet_bootstrap.cc:654] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:26.192256 4135 log.cc:824] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:26.192504 4134 log.cc:824] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:26.194691 4136 tablet_bootstrap.cc:492] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Bootstrap starting.
I20260808 23:43:26.195246 4135 tablet_bootstrap.cc:492] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: No bootstrap required, opened a new log
I20260808 23:43:26.195588 4135 ts_tablet_manager.cc:1397] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 23:43:26.198122 4134 tablet_bootstrap.cc:492] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: No bootstrap required, opened a new log
I20260808 23:43:26.198194 4136 tablet_bootstrap.cc:654] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:26.198400 4134 ts_tablet_manager.cc:1397] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260808 23:43:26.199518 4136 log.cc:824] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:26.202468 4136 tablet_bootstrap.cc:492] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: No bootstrap required, opened a new log
I20260808 23:43:26.202733 4136 ts_tablet_manager.cc:1397] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 23:43:26.203320 4135 raft_consensus.cc:348] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.203763 4135 raft_consensus.cc:374] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:26.203929 4135 raft_consensus.cc:729] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c5490bf8733411f9f590e08eaf22f07, State: Initialized, Role: FOLLOWER
I20260808 23:43:26.204727 4135 consensus_queue.cc:260] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.206634 4135 ts_tablet_manager.cc:1428] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260808 23:43:26.208804 4134 raft_consensus.cc:348] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.209228 4134 raft_consensus.cc:374] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:26.209331 4134 raft_consensus.cc:729] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09864c13a4e74fc6b3f55b82448b112c, State: Initialized, Role: FOLLOWER
I20260808 23:43:26.210022 4134 consensus_queue.cc:260] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.211176 4113 heartbeater.cc:503] Master 127.18.186.190:42427 was elected leader, sending a full tablet report...
I20260808 23:43:26.211898 4134 ts_tablet_manager.cc:1428] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260808 23:43:26.213042 4136 raft_consensus.cc:348] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.213388 4136 raft_consensus.cc:374] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:26.213493 4136 raft_consensus.cc:729] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c51a5b5485804b7ba7c1736e0b0cd6f9, State: Initialized, Role: FOLLOWER
I20260808 23:43:26.214103 4136 consensus_queue.cc:260] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.215313 3969 heartbeater.cc:503] Master 127.18.186.190:42427 was elected leader, sending a full tablet report...
I20260808 23:43:26.215909 4136 ts_tablet_manager.cc:1428] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Time spent starting tablet: real 0.013s user 0.009s sys 0.006s
W20260808 23:43:26.264293 3831 tablet.cc:2369] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:43:26.268193 4140 raft_consensus.cc:482] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:43:26.268507 4140 raft_consensus.cc:504] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.270192 4140 leader_election.cc:288] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c51a5b5485804b7ba7c1736e0b0cd6f9 (127.18.186.130:44011), 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605)
I20260808 23:43:26.277102 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.270807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:49073 (local address 127.18.186.131:36605)
0808 23:43:26.271034 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.271040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.271375 (+ 335us) server_negotiation.cc:407] Connection header received
0808 23:43:26.271639 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.271646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.271727 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.271841 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:26.272578 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.273383 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.275321 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.276113 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.276231 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.276309 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.276702 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.276851 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 23:43:26.277196 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.270476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:36605 (local address 127.18.186.129:49073)
0808 23:43:26.271225 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:26.271253 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:26.271475 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:26.271964 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:26.271969 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:26.271997 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:26.272423 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.272435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.273522 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.273527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.275166 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.275178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.276326 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.276332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.276519 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.276543 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:43:26.276660 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 23:43:26.276794 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":48,"thread_start_us":354,"threads_started":1}
I20260808 23:43:26.277256 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.270684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:46185 (local address 127.18.186.130:44011)
0808 23:43:26.270936 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.270945 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.271861 (+ 916us) server_negotiation.cc:407] Connection header received
0808 23:43:26.271996 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.272001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.272078 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.272180 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:26.272787 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.273539 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.275381 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.276222 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.276380 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.276454 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.276858 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.277001 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:43:26.277904 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.270493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:44011 (local address 127.18.186.129:46185)
0808 23:43:26.271727 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:26.271745 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:26.271869 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:26.272203 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:26.272206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:26.272226 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:26.272611 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.272625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.273682 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.273686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.275203 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:26.275215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.276471 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.276477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:26.276673 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.276707 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:43:26.277404 (+ 697us) client_negotiation.cc:240] Negotiation successful
0808 23:43:26.277531 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":547,"thread_start_us":333,"threads_started":1}
I20260808 23:43:26.278465 4065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a9f34946894221b70d73ddb12bcc0f" candidate_uuid: "8c5490bf8733411f9f590e08eaf22f07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09864c13a4e74fc6b3f55b82448b112c" is_pre_election: true
I20260808 23:43:26.278980 4065 raft_consensus.cc:2454] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c5490bf8733411f9f590e08eaf22f07 in term 0.
I20260808 23:43:26.278955 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a9f34946894221b70d73ddb12bcc0f" candidate_uuid: "8c5490bf8733411f9f590e08eaf22f07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" is_pre_election: true
I20260808 23:43:26.279462 3922 raft_consensus.cc:2454] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c5490bf8733411f9f590e08eaf22f07 in term 0.
I20260808 23:43:26.279727 3706 leader_election.cc:302] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09864c13a4e74fc6b3f55b82448b112c, 8c5490bf8733411f9f590e08eaf22f07; no voters:
I20260808 23:43:26.280179 4140 raft_consensus.cc:2781] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:43:26.280339 4140 raft_consensus.cc:482] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:43:26.280478 4140 raft_consensus.cc:3037] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:26.282637 4140 raft_consensus.cc:504] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.283242 4140 leader_election.cc:288] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [CANDIDATE]: Term 1 election: Requested vote from peers c51a5b5485804b7ba7c1736e0b0cd6f9 (127.18.186.130:44011), 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605)
I20260808 23:43:26.283730 4065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a9f34946894221b70d73ddb12bcc0f" candidate_uuid: "8c5490bf8733411f9f590e08eaf22f07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09864c13a4e74fc6b3f55b82448b112c"
I20260808 23:43:26.283746 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85a9f34946894221b70d73ddb12bcc0f" candidate_uuid: "8c5490bf8733411f9f590e08eaf22f07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9"
I20260808 23:43:26.283921 4065 raft_consensus.cc:3037] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:26.283932 3922 raft_consensus.cc:3037] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:26.286190 4065 raft_consensus.cc:2454] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c5490bf8733411f9f590e08eaf22f07 in term 1.
I20260808 23:43:26.286286 3922 raft_consensus.cc:2454] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c5490bf8733411f9f590e08eaf22f07 in term 1.
I20260808 23:43:26.286630 3706 leader_election.cc:302] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09864c13a4e74fc6b3f55b82448b112c, 8c5490bf8733411f9f590e08eaf22f07; no voters:
I20260808 23:43:26.286944 4140 raft_consensus.cc:2781] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:43:26.287400 4140 raft_consensus.cc:686] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 1 LEADER]: Becoming Leader. State: Replica: 8c5490bf8733411f9f590e08eaf22f07, State: Running, Role: LEADER
I20260808 23:43:26.287917 4140 consensus_queue.cc:237] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } }
I20260808 23:43:26.292110 3620 catalog_manager.cc:5495] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c5490bf8733411f9f590e08eaf22f07 (127.18.186.129). New cstate: current_term: 1 leader_uuid: "8c5490bf8733411f9f590e08eaf22f07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } health_report { overall_health: UNKNOWN } } }
W20260808 23:43:26.299671 3971 tablet.cc:2369] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:43:26.320250 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.314195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.18.186.129:38707)
0808 23:43:26.314405 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.314411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.314521 (+ 110us) server_negotiation.cc:407] Connection header received
0808 23:43:26.314770 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.314775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.314899 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.315069 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:43:26.315503 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.316633 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.318229 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.318693 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.318786 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.319261 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:43:26.319800 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:43:26.319931 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.320001 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.320063 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 23:43:26.326938 4114 tablet.cc:2369] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:43:26.328339 3922 raft_consensus.cc:1261] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 FOLLOWER]: Refusing update from remote peer 8c5490bf8733411f9f590e08eaf22f07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:43:26.328356 4065 raft_consensus.cc:1261] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 1 FOLLOWER]: Refusing update from remote peer 8c5490bf8733411f9f590e08eaf22f07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:43:26.329241 4145 consensus_queue.cc:1035] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:43:26.329638 4140 consensus_queue.cc:1035] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:43:26.345340 4150 mvcc.cc:205] Tried to move back new op lower bound from 7316408755511078912 to 7316408755358621696. Current Snapshot: MvccSnapshot[applied={T|T < 7316408755511078912}]
I20260808 23:43:26.358359 3622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:43:26.358659 3622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:43:26.362459 3622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Sending DeleteTablet for 3 replicas of tablet 85a9f34946894221b70d73ddb12bcc0f
I20260808 23:43:26.363538 3755 tablet_service.cc:1514] Processing DeleteTablet for tablet 85a9f34946894221b70d73ddb12bcc0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:26 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:57940
I20260808 23:43:26.363643 3901 tablet_service.cc:1514] Processing DeleteTablet for tablet 85a9f34946894221b70d73ddb12bcc0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:26 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:44054
I20260808 23:43:26.364538 4044 tablet_service.cc:1514] Processing DeleteTablet for tablet 85a9f34946894221b70d73ddb12bcc0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:26 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:57736
I20260808 23:43:26.365799 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 3687
I20260808 23:43:26.391177 4162 tablet_replica.cc:330] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: stopping tablet replica
I20260808 23:43:26.393513 4160 tablet_replica.cc:330] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: stopping tablet replica
I20260808 23:43:26.393639 4162 raft_consensus.cc:2229] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:43:26.394210 4162 raft_consensus.cc:2258] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:26.394714 4160 raft_consensus.cc:2229] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:43:26.395099 4161 tablet_replica.cc:330] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: stopping tablet replica
I20260808 23:43:26.395612 4161 raft_consensus.cc:2229] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:43:26.395674 4160 raft_consensus.cc:2258] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:26.396186 4161 raft_consensus.cc:2258] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:26.397358 4162 ts_tablet_manager.cc:1905] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:26.397941 4160 ts_tablet_manager.cc:1905] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:26.405462 4162 ts_tablet_manager.cc:1918] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:26.405691 4162 log.cc:1196] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/85a9f34946894221b70d73ddb12bcc0f
I20260808 23:43:26.406281 4162 ts_tablet_manager.cc:1939] T 85a9f34946894221b70d73ddb12bcc0f P 09864c13a4e74fc6b3f55b82448b112c: Deleting consensus metadata
I20260808 23:43:26.407125 4160 ts_tablet_manager.cc:1918] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:26.407373 4160 log.cc:1196] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/85a9f34946894221b70d73ddb12bcc0f
I20260808 23:43:26.408002 4160 ts_tablet_manager.cc:1939] T 85a9f34946894221b70d73ddb12bcc0f P 8c5490bf8733411f9f590e08eaf22f07: Deleting consensus metadata
I20260808 23:43:26.408444 3605 catalog_manager.cc:4841] TS 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605): tablet 85a9f34946894221b70d73ddb12bcc0f (table test-table [id=c4a4f7c3466d499b84b1f56108ab91d3]) successfully deleted
I20260808 23:43:26.409884 3604 catalog_manager.cc:4841] TS 8c5490bf8733411f9f590e08eaf22f07 (127.18.186.129:38707): tablet 85a9f34946894221b70d73ddb12bcc0f (table test-table [id=c4a4f7c3466d499b84b1f56108ab91d3]) successfully deleted
I20260808 23:43:26.410317 4161 ts_tablet_manager.cc:1905] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 23:43:26 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:43:26.417454 4161 ts_tablet_manager.cc:1918] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:26.417630 4161 log.cc:1196] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/85a9f34946894221b70d73ddb12bcc0f
I20260808 23:43:26.418169 4161 ts_tablet_manager.cc:1939] T 85a9f34946894221b70d73ddb12bcc0f P c51a5b5485804b7ba7c1736e0b0cd6f9: Deleting consensus metadata
I20260808 23:43:26.420073 3606 catalog_manager.cc:4841] TS c51a5b5485804b7ba7c1736e0b0cd6f9 (127.18.186.130:44011): tablet 85a9f34946894221b70d73ddb12bcc0f (table test-table [id=c4a4f7c3466d499b84b1f56108ab91d3]) successfully deleted
I20260808 23:43:26.421936 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.367484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57952 (local address 127.18.186.129:38707)
0808 23:43:26.367698 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.367706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.389576 (+ 21870us) server_negotiation.cc:407] Connection header received
0808 23:43:26.390070 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.390076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.390172 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.390244 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:26.393430 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.394804 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.398589 (+ 3785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.399288 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.399389 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.415241 (+ 15852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:26.415259 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:26.415321 (+ 62us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:26.415386 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:26.418919 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.419747 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.419754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.419756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.419959 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.420483 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.420486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.420488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.420834 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:26.421035 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.421507 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.421677 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 23:43:26.422751 3824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:26.525990 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 3687
I20260808 23:43:26.550508 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 3836
Aug 08 23:43:26 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:43:26.567294 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.551379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44068 (local address 127.18.186.130:44011)
0808 23:43:26.551573 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.551577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.551595 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:43:26.551774 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.551778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.551850 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.551954 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:26.553316 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.554104 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.555543 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.556197 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.556325 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.560404 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:26.560417 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:26.560435 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:26.560483 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:26.564553 (+ 4070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.565228 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.565243 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.565248 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.565394 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.565731 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.565736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.565740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.566225 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:26.566522 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.566714 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.566924 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 23:43:26.568068 3964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:26.659508 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 3836
I20260808 23:43:26.679224 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 3976
Aug 08 23:43:26 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:43:26.694379 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:26.680110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.18.186.131:36605)
0808 23:43:26.680296 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:26.680301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:26.680320 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:43:26.680456 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:26.680459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:26.680524 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:26.680606 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:26.681872 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.682596 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.684358 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:26.684753 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:26.684909 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:26.689331 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:26.689345 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:26.689361 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:26.689400 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:26.692100 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.692709 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.692716 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.692718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.692844 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:26.693152 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:26.693155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:26.693157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:26.693499 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:26.693710 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:26.693948 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 23:43:26.694117 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:43:26.695086 4108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:26.784900 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 3976
I20260808 23:43:26.805336 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 3585
I20260808 23:43:26.806604 3655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:26.924435 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 3585
I20260808 23:43:46.950111 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42427
--webserver_interface=127.18.186.190
--webserver_port=43799
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:43:47.290717 4168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:47.291052 4168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:47.291215 4168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:47.298482 4168 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:43:47.298617 4168 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:43:47.298712 4168 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:43:47.298815 4168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:43:47.298924 4168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:47.299045 4168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:47.299130 4168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:43:47.299211 4168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:43:47.309096 4168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:47.310449 4168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:47.311880 4168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:47.338414 4174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:47.338528 4175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:47.338435 4177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:47.339325 4168 server_base.cc:1034] running on GCE node
I20260808 23:43:47.340330 4168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:47.341920 4168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:47.343151 4168 hybrid_clock.cc:648] HybridClock initialized: now 1786232627343067 us; error 56 us; skew 500 ppm
I20260808 23:43:47.343613 4168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:47 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232627, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:47.352133 4168 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:43:47.353756 4168 webserver.cc:460] Webserver started at http://127.18.186.190:43799/ using document root <none> and password file <none>
I20260808 23:43:47.354518 4168 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:47.354626 4168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:47.360657 4168 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 23:43:47.363752 4184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:47.365128 4168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:43:47.365267 4168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "36a7abc22b4f458b8504f82c27c07375"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "1a24c49f70df55cf0eca9aada240e2b9"
server_key_iv: "69298c773cd1ade87c61577f093f5ee7"
server_key_version: "encryptionkey@0"
I20260808 23:43:47.366278 4168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:47.384024 4168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:47.388005 4168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:47.388248 4168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:47.408541 4168 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42427
I20260808 23:43:47.408592 4238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42427 every 8 connection(s)
I20260808 23:43:47.410223 4168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:43:47.417520 4239 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:43:47.420301 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 4168
I20260808 23:43:47.420930 4239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Bootstrap starting.
I20260808 23:43:47.421209 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:38707
--local_ip_for_outbound_sockets=127.18.186.129
--tserver_master_addrs=127.18.186.190:42427
--webserver_port=33801
--webserver_interface=127.18.186.129
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:43:47.444458 4239 log.cc:824] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:47.461769 4239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:43:47.462450 4239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Bootstrap complete.
I20260808 23:43:47.476553 4239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:43:47.477806 4239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a7abc22b4f458b8504f82c27c07375, State: Initialized, Role: FOLLOWER
I20260808 23:43:47.478562 4239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:43:47.478849 4239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:43:47.478973 4239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:43:47.479146 4239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:43:47.485141 4239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:43:47.485803 4239 leader_election.cc:302] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a7abc22b4f458b8504f82c27c07375; no voters:
I20260808 23:43:47.486522 4239 leader_election.cc:288] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:43:47.486693 4245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:43:47.488059 4245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 2 LEADER]: Becoming Leader. State: Replica: 36a7abc22b4f458b8504f82c27c07375, State: Running, Role: LEADER
I20260808 23:43:47.488817 4239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:43:47.488730 4245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:43:47.491067 4247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a7abc22b4f458b8504f82c27c07375. Latest consensus state: current_term: 2 leader_uuid: "36a7abc22b4f458b8504f82c27c07375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } } }
I20260808 23:43:47.491874 4247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: This master's current role is: LEADER
I20260808 23:43:47.491458 4246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "36a7abc22b4f458b8504f82c27c07375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } } }
I20260808 23:43:47.492506 4246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: This master's current role is: LEADER
I20260808 23:43:47.496104 4251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:43:47.504491 4251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:43:47.505959 4251 catalog_manager.cc:1259] Loaded cluster ID: bbe6cd9a9b86495abee67736a9ddab7b
I20260808 23:43:47.506104 4251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:43:47.510195 4251 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:43:47.511865 4251 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Loaded TSK: 0
I20260808 23:43:47.542558 4264 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 23:43:47.549090 4251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Generated new TSK 1
I20260808 23:43:47.549494 4251 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:43:47.838618 4242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:47.838919 4242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:47.839066 4242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:47.846436 4242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:47.846553 4242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:47.846750 4242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:43:47.856415 4242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:38707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=33801
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:47.857661 4242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:47.858975 4242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:47.873473 4270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:47.873442 4274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:47.873446 4272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:47.874491 4242 server_base.cc:1034] running on GCE node
I20260808 23:43:47.875061 4242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:47.876008 4242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:47.877246 4242 hybrid_clock.cc:648] HybridClock initialized: now 1786232627877184 us; error 54 us; skew 500 ppm
I20260808 23:43:47.877583 4242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:47 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232627, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:47.880803 4242 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:43:47.882344 4242 webserver.cc:460] Webserver started at http://127.18.186.129:33801/ using document root <none> and password file <none>
I20260808 23:43:47.883037 4242 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:47.883185 4242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:47.888571 4242 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 23:43:47.891706 4282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:47.893194 4242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:43:47.893384 4242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "8c5490bf8733411f9f590e08eaf22f07"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "d81a81a3d74d446018ae7e97e4a23f45"
server_key_iv: "35f5209dbb8bb0dfdc2a5783cb702c54"
server_key_version: "encryptionkey@0"
I20260808 23:43:47.894331 4242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:47.919633 4242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:47.923981 4242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:47.924259 4242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:47.925259 4242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:47.926568 4242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:47.926692 4242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:47.926827 4242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:47.926915 4242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:47.960114 4242 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:38707
I20260808 23:43:47.960201 4406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:38707 every 8 connection(s)
I20260808 23:43:47.961536 4242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:43:47.965893 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 4242
I20260808 23:43:47.966536 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:44011
--local_ip_for_outbound_sockets=127.18.186.130
--tserver_master_addrs=127.18.186.190:42427
--webserver_port=39021
--webserver_interface=127.18.186.130
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:43:47 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232627, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:47.987512 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:47.964713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:42197)
0808 23:43:47.965592 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:47.965635 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:47.967528 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:47.969105 (+ 1577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:47.969113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:47.969723 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:47.970321 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:47.970338 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:47.972819 (+ 2481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:47.972823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:47.974035 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:47.974047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:47.975521 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:47.975527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:47.975721 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:47.976400 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:47.976422 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:47.978688 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:47.983400 (+ 4712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:47.983407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:47.983417 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:47.983918 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:47.984398 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:47.984401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:47.984403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:47.984589 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:47.985587 (+ 998us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:47.985592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:47.985864 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 23:43:47.986074 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 23:43:47.986440 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":348,"threads_started":1}
I20260808 23:43:47.988170 4409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:47.964393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:42197 (local address 127.18.186.190:42427)
0808 23:43:47.964991 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:47.965002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:47.965786 (+ 784us) server_negotiation.cc:407] Connection header received
0808 23:43:47.968148 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:47.968170 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:47.968607 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:47.968995 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:47.970567 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:47.972632 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:47.974179 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:47.975266 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:47.975501 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:47.978966 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:47.978978 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:47.979000 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:47.979051 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:47.983261 (+ 4210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:47.984054 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:47.984065 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:47.984076 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:47.984288 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:47.984726 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:47.984734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:47.984740 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:47.985461 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:47.985657 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:47.986386 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 23:43:47.986887 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":251,"threads_started":1}
I20260808 23:43:47.991036 4407 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42427
I20260808 23:43:47.991374 4407 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:47.992288 4407 heartbeater.cc:511] Master 127.18.186.190:42427 requested a full tablet report, sending...
I20260808 23:43:47.994757 4202 ts_manager.cc:194] Registered new tserver with Master: 8c5490bf8733411f9f590e08eaf22f07 (127.18.186.129:38707)
I20260808 23:43:47.996461 4202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:42197
W20260808 23:43:48.309437 4411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:48.309710 4411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:48.309894 4411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:48.317214 4411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:48.317344 4411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:48.317545 4411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:43:48.326855 4411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:44011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=39021
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:48.328055 4411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:48.329298 4411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:48.342430 4419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:48.342519 4416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:48.342494 4417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:48.343613 4411 server_base.cc:1034] running on GCE node
I20260808 23:43:48.344169 4411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:48.345082 4411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:48.346410 4411 hybrid_clock.cc:648] HybridClock initialized: now 1786232628346339 us; error 60 us; skew 500 ppm
I20260808 23:43:48.346735 4411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:48 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232628, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:48.349869 4411 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:43:48.351271 4411 webserver.cc:460] Webserver started at http://127.18.186.130:39021/ using document root <none> and password file <none>
I20260808 23:43:48.352016 4411 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:48.352185 4411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:48.357362 4411 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 23:43:48.360026 4426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:48.361261 4411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:43:48.361454 4411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "a66f85187830733e639782abfb301f68"
server_key_iv: "253d18886a5891a886b90ff147894090"
server_key_version: "encryptionkey@0"
I20260808 23:43:48.362404 4411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:48.403360 4411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:48.407366 4411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:48.407681 4411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:48.408803 4411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:48.410193 4411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:48.410332 4411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:48.410457 4411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:48.410547 4411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:48.445956 4411 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:44011
I20260808 23:43:48.446048 4544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:44011 every 8 connection(s)
I20260808 23:43:48.447463 4411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:43:48.450111 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 4411
I20260808 23:43:48.450755 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:36605
--local_ip_for_outbound_sockets=127.18.186.131
--tserver_master_addrs=127.18.186.190:42427
--webserver_port=33565
--webserver_interface=127.18.186.131
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:43:48 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232628, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:48.471622 4409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.450104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:59107 (local address 127.18.186.190:42427)
0808 23:43:48.450329 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:48.450336 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:48.451630 (+ 1294us) server_negotiation.cc:407] Connection header received
0808 23:43:48.454519 (+ 2889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:48.454524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:48.454608 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:48.454698 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:48.456811 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.457524 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.460037 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.460457 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.460565 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:48.464710 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:48.464719 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:48.464728 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:48.464790 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:48.468172 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:48.469132 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:48.469136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:48.469138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:48.469276 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:48.469746 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:48.469750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:48.469752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:48.470136 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:48.470294 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:48.470990 (+ 696us) server_negotiation.cc:299] Negotiation successful
0808 23:43:48.471230 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 23:43:48.472577 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.450557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:59107)
0808 23:43:48.451405 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:48.451490 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:48.454211 (+ 2721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:48.454977 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:48.454991 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:48.455803 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:48.456618 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:48.456647 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.457686 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.457695 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:48.459880 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:48.459897 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.460682 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.460691 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:48.461048 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:48.461786 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:48.461820 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:48.464455 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:48.468326 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:48.468338 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:48.468356 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:48.469010 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:48.469389 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:48.469394 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:48.469402 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:48.469617 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:48.470272 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:48.470279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:48.470621 (+ 342us) client_negotiation.cc:769] Sending connection context
0808 23:43:48.470887 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 23:43:48.471461 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":331,"threads_started":1}
I20260808 23:43:48.474056 4545 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42427
I20260808 23:43:48.474382 4545 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:48.475132 4545 heartbeater.cc:511] Master 127.18.186.190:42427 requested a full tablet report, sending...
I20260808 23:43:48.476665 4202 ts_manager.cc:194] Registered new tserver with Master: c51a5b5485804b7ba7c1736e0b0cd6f9 (127.18.186.130:44011)
I20260808 23:43:48.477558 4202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:59107
W20260808 23:43:48.801118 4548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:43:48.801397 4548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:43:48.801491 4548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:43:48.808642 4548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:43:48.808748 4548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:43:48.808858 4548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:43:48.818286 4548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:36605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=33565
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:43:48.819664 4548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:43:48.820905 4548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:43:48.834163 4556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:48.834208 4553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:43:48.834847 4554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:43:48.834995 4548 server_base.cc:1034] running on GCE node
I20260808 23:43:48.835687 4548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:43:48.836650 4548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:43:48.837888 4548 hybrid_clock.cc:648] HybridClock initialized: now 1786232628837829 us; error 47 us; skew 500 ppm
I20260808 23:43:48.838204 4548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:43:48 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232628, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:43:48.841333 4548 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:43:48.842828 4548 webserver.cc:460] Webserver started at http://127.18.186.131:33565/ using document root <none> and password file <none>
I20260808 23:43:48.843597 4548 fs_manager.cc:359] Metadata directory not provided
I20260808 23:43:48.843760 4548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:43:48.849035 4548 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:43:48.851809 4563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:48.853173 4548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:43:48.853370 4548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "09864c13a4e74fc6b3f55b82448b112c"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "1648097478eb6549e0b73305d9307ebd"
server_key_iv: "6f03ef7f05ce1881605ec4c91b2235f5"
server_key_version: "encryptionkey@0"
I20260808 23:43:48.854349 4548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:43:48.868367 4548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:43:48.874342 4548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:43:48.874673 4548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:43:48.875805 4548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:43:48.877785 4548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:43:48.877900 4548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:48.878034 4548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:43:48.878137 4548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:43:48.915894 4548 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:36605
I20260808 23:43:48.916042 4685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:36605 every 8 connection(s)
I20260808 23:43:48.917382 4548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:43:48.924973 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 4548
Aug 08 23:43:48 dist-test-slave-5k54 krb5kdc[3571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232628, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:43:48.938922 4409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.919983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:56291 (local address 127.18.186.190:42427)
0808 23:43:48.920162 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:48.920166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:48.921340 (+ 1174us) server_negotiation.cc:407] Connection header received
0808 23:43:48.923091 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:48.923096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:48.923162 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:48.923251 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:48.924754 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.925619 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.927753 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.928521 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.928654 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:48.932553 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:48.932558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:48.932561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:48.932592 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:48.935638 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:48.936527 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:48.936531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:48.936533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:48.936668 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:48.937213 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:48.937217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:48.937219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:48.937499 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:48.937674 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:48.938336 (+ 662us) server_negotiation.cc:299] Negotiation successful
0808 23:43:48.938547 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:43:48.939970 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.920502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:56291)
0808 23:43:48.921174 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:48.921210 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:48.922840 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:48.923510 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:48.923518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:48.924018 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:43:48.924588 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:48.924605 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.925856 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.925860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:48.927609 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:48.927621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.928730 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.928735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:48.928924 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:48.929615 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:43:48.929643 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:43:48.932347 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:43:48.935783 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:48.935792 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:48.935807 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:48.936382 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:48.936807 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:43:48.936814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:43:48.936820 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:43:48.937069 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:43:48.937639 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:43:48.937648 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:43:48.937992 (+ 344us) client_negotiation.cc:769] Sending connection context
0808 23:43:48.938241 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 23:43:48.938758 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":260,"threads_started":1}
I20260808 23:43:48.941282 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.925975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55024 (local address 127.18.186.190:42427)
0808 23:43:48.926477 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:48.926483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:48.926826 (+ 343us) server_negotiation.cc:407] Connection header received
0808 23:43:48.926950 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:48.926955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:48.927019 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:48.927134 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:48.929107 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.930205 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.932035 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.932451 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.932589 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:48.935362 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:48.935367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:48.935371 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:48.935399 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:48.938148 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:48.938826 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:48.938832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:48.938836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:48.938982 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:48.939528 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:48.939534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:48.939538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:48.939789 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:48.940056 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:48.940746 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 23:43:48.940966 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":244,"threads_started":1}
I20260808 23:43:48.941488 4687 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42427
I20260808 23:43:48.941800 4687 heartbeater.cc:464] Registering TS with master...
I20260808 23:43:48.942646 4687 heartbeater.cc:511] Master 127.18.186.190:42427 requested a full tablet report, sending...
I20260808 23:43:48.944028 4202 ts_manager.cc:194] Registered new tserver with Master: 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605)
I20260808 23:43:48.944839 4202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:56291
I20260808 23:43:48.955833 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:43:48.963040 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.956721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55038 (local address 127.18.186.190:42427)
0808 23:43:48.956903 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:48.956910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:48.957461 (+ 551us) server_negotiation.cc:407] Connection header received
0808 23:43:48.957643 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:48.957647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:48.957771 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:48.957894 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:43:48.958327 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.958998 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.961147 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.961577 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.961669 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:48.962304 (+ 635us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:43:48.962568 (+ 264us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 23:43:48.962840 (+ 272us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55038: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 23:43:48.963186 4690 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55038: token expired
W20260808 23:43:48.964015 4119 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.186.190:42427)
I20260808 23:43:48.977056 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.964494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.18.186.190:42427)
0808 23:43:48.964674 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:48.964682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:48.964700 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:43:48.964903 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:48.964907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:48.964959 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:48.965059 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:48.966373 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.967107 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.968514 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:48.968886 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:48.969022 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:48.971907 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:48.971912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:48.971915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:48.971944 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:48.974376 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:48.975023 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:48.975030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:48.975034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:48.975170 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:48.975628 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:48.975633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:48.975636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:48.975947 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:48.976201 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:48.976500 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 23:43:48.976709 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:43:48.980240 4202 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:43:48.987509 4202 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:43:49.000581 4407 heartbeater.cc:503] Master 127.18.186.190:42427 was elected leader, sending a full tablet report...
I20260808 23:43:49.032534 4699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.998823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:38707 (local address 127.0.0.1:60996)
0808 23:43:49.000813 (+ 1990us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:49.000861 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:49.001052 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:49.020203 (+ 19151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:49.020212 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:49.020252 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:49.020849 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.020869 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.023606 (+ 2737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.023615 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.027183 (+ 3568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.027208 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.031160 (+ 3952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.031170 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.031488 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.031575 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 23:43:49.031717 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 23:43:49.032015 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1665,"threads_started":1}
I20260808 23:43:49.034683 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.998926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.18.186.129:38707)
0808 23:43:49.018048 (+ 19122us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.018063 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.018108 (+ 45us) server_negotiation.cc:407] Connection header received
0808 23:43:49.018266 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.018279 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.018625 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.020181 (+ 1556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:49.021143 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.023345 (+ 2202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.028635 (+ 5290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.030796 (+ 2161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.032597 (+ 1801us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.032798 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.032868 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.034451 (+ 1583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18901,"thread_start_us":20188,"threads_started":1}
I20260808 23:43:49.040086 4336 tablet_service.cc:1467] Processing CreateTablet for tablet f6d37466f87d4a34863564223afa845b (DEFAULT_TABLE table=test-table [id=5737875fa2364bad8b8eae50145e9c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:49.041710 4336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6d37466f87d4a34863564223afa845b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:49.043208 4699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.999034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:36605 (local address 127.0.0.1:36406)
0808 23:43:49.033074 (+ 34040us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:49.033095 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:49.033198 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:49.036141 (+ 2943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:49.036145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:49.036173 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:49.036383 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.036403 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.038615 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.038621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.040653 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.040668 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.042449 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.042455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.042636 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.042669 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:43:49.042740 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 23:43:49.042863 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33910,"mutex_wait_us":77,"thread_start_us":1284,"threads_started":1}
I20260808 23:43:49.044628 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:48.999800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.18.186.131:36605)
0808 23:43:49.035498 (+ 35698us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.035509 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.035548 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:43:49.035651 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.035659 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.035920 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.036560 (+ 640us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:49.036867 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.038457 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.040852 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.042254 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.043882 (+ 1628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.044034 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.044105 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.044255 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35476,"thread_start_us":35297,"threads_started":1}
I20260808 23:43:49.044845 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.002156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:44011 (local address 127.0.0.1:59302)
0808 23:43:49.034623 (+ 32467us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:49.034646 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:49.034810 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:49.035795 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:49.035803 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:49.035841 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:49.036276 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.036291 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.038785 (+ 2494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.038792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.041355 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.041374 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.043968 (+ 2594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.043976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.044187 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.044234 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 23:43:49.044321 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 23:43:49.044441 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32307,"mutex_wait_us":26,"thread_start_us":30123,"threads_started":1}
I20260808 23:43:49.045550 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.002799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59302 (local address 127.18.186.130:44011)
0808 23:43:49.033640 (+ 30841us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.033653 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.034785 (+ 1132us) server_negotiation.cc:407] Connection header received
0808 23:43:49.034988 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.034998 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.035286 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.035523 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:49.036514 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.038590 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.041562 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.043266 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.043487 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.045057 (+ 1570us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.045124 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.045306 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30615,"thread_start_us":31311,"threads_started":1}
I20260808 23:43:49.049166 4614 tablet_service.cc:1467] Processing CreateTablet for tablet f6d37466f87d4a34863564223afa845b (DEFAULT_TABLE table=test-table [id=5737875fa2364bad8b8eae50145e9c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:49.049142 4478 tablet_service.cc:1467] Processing CreateTablet for tablet f6d37466f87d4a34863564223afa845b (DEFAULT_TABLE table=test-table [id=5737875fa2364bad8b8eae50145e9c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:43:49.050534 4478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6d37466f87d4a34863564223afa845b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:49.050551 4614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6d37466f87d4a34863564223afa845b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:43:49.054805 4703 tablet_bootstrap.cc:492] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Bootstrap starting.
I20260808 23:43:49.058472 4703 tablet_bootstrap.cc:654] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:49.059729 4703 log.cc:824] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:49.062867 4703 tablet_bootstrap.cc:492] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: No bootstrap required, opened a new log
I20260808 23:43:49.063158 4703 ts_tablet_manager.cc:1397] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 23:43:49.066597 4706 tablet_bootstrap.cc:492] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Bootstrap starting.
I20260808 23:43:49.067718 4705 tablet_bootstrap.cc:492] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Bootstrap starting.
I20260808 23:43:49.070783 4706 tablet_bootstrap.cc:654] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:49.070938 4705 tablet_bootstrap.cc:654] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Neither blocks nor log segments found. Creating new log.
I20260808 23:43:49.072008 4705 log.cc:824] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:49.072093 4706 log.cc:824] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:43:49.073290 4703 raft_consensus.cc:348] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.073850 4703 raft_consensus.cc:374] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:49.074002 4703 raft_consensus.cc:729] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c5490bf8733411f9f590e08eaf22f07, State: Initialized, Role: FOLLOWER
I20260808 23:43:49.074985 4705 tablet_bootstrap.cc:492] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: No bootstrap required, opened a new log
I20260808 23:43:49.074908 4703 consensus_queue.cc:260] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.075275 4705 ts_tablet_manager.cc:1397] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 23:43:49.077234 4703 ts_tablet_manager.cc:1428] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260808 23:43:49.079928 4706 tablet_bootstrap.cc:492] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: No bootstrap required, opened a new log
I20260808 23:43:49.080196 4706 ts_tablet_manager.cc:1397] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.001s
I20260808 23:43:49.085909 4705 raft_consensus.cc:348] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.086270 4705 raft_consensus.cc:374] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:49.086370 4705 raft_consensus.cc:729] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c51a5b5485804b7ba7c1736e0b0cd6f9, State: Initialized, Role: FOLLOWER
I20260808 23:43:49.086920 4705 consensus_queue.cc:260] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.088002 4545 heartbeater.cc:503] Master 127.18.186.190:42427 was elected leader, sending a full tablet report...
I20260808 23:43:49.087981 4706 raft_consensus.cc:348] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.088366 4706 raft_consensus.cc:374] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:43:49.088512 4706 raft_consensus.cc:729] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09864c13a4e74fc6b3f55b82448b112c, State: Initialized, Role: FOLLOWER
I20260808 23:43:49.088593 4705 ts_tablet_manager.cc:1428] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Time spent starting tablet: real 0.013s user 0.004s sys 0.006s
I20260808 23:43:49.089133 4706 consensus_queue.cc:260] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.092788 4706 ts_tablet_manager.cc:1428] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260808 23:43:49.093011 4687 heartbeater.cc:503] Master 127.18.186.190:42427 was elected leader, sending a full tablet report...
W20260808 23:43:49.169008 4688 tablet.cc:2369] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:43:49.199545 4546 tablet.cc:2369] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:43:49.213930 4408 tablet.cc:2369] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:43:49.374003 4710 raft_consensus.cc:482] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:43:49.374344 4710 raft_consensus.cc:504] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.376120 4710 leader_election.cc:288] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c5490bf8733411f9f590e08eaf22f07 (127.18.186.129:38707), 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605)
I20260808 23:43:49.385723 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.376597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:42563 (local address 127.18.186.131:36605)
0808 23:43:49.376790 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.376797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.377573 (+ 776us) server_negotiation.cc:407] Connection header received
0808 23:43:49.377801 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.377806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.377886 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.377995 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:49.378860 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.379995 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.383045 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.384343 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.384514 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.384578 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.385324 (+ 746us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.385469 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:43:49.385726 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.376700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:49707 (local address 127.18.186.129:38707)
0808 23:43:49.376898 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.376905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.377635 (+ 730us) server_negotiation.cc:407] Connection header received
0808 23:43:49.377804 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.377810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.377886 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.377995 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:43:49.378861 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.379995 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.383045 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.384312 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.384503 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.384578 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.385341 (+ 763us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.385473 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:43:49.385985 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.376417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:36605 (local address 127.18.186.130:42563)
0808 23:43:49.377434 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:49.377460 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:49.377645 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:49.378074 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:49.378079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:49.378128 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:49.378724 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.378733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.380142 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.380147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.382864 (+ 2717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.382879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.384784 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.384794 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.385114 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.385162 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 23:43:49.385264 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 23:43:49.385451 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":341,"spinlock_wait_cycles":6400,"thread_start_us":668,"threads_started":1}
I20260808 23:43:49.386011 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.376515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:38707 (local address 127.18.186.130:49707)
0808 23:43:49.377521 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0808 23:43:49.377537 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:43:49.377649 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:43:49.378074 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:43:49.378079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:43:49.378119 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:43:49.378667 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.378683 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.380143 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.380147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.382863 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:43:49.382879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.384818 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.384823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:43:49.385109 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.385169 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 23:43:49.385264 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 23:43:49.385451 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":237,"spinlock_wait_cycles":3584,"thread_start_us":475,"threads_started":1}
I20260808 23:43:49.387028 4358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6d37466f87d4a34863564223afa845b" candidate_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c5490bf8733411f9f590e08eaf22f07" is_pre_election: true
I20260808 23:43:49.387106 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6d37466f87d4a34863564223afa845b" candidate_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09864c13a4e74fc6b3f55b82448b112c" is_pre_election: true
I20260808 23:43:49.387598 4636 raft_consensus.cc:2454] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c51a5b5485804b7ba7c1736e0b0cd6f9 in term 0.
I20260808 23:43:49.387619 4358 raft_consensus.cc:2454] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c51a5b5485804b7ba7c1736e0b0cd6f9 in term 0.
I20260808 23:43:49.388271 4428 leader_election.cc:302] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09864c13a4e74fc6b3f55b82448b112c, 8c5490bf8733411f9f590e08eaf22f07, c51a5b5485804b7ba7c1736e0b0cd6f9; no voters:
I20260808 23:43:49.388744 4710 raft_consensus.cc:2781] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:43:49.388908 4710 raft_consensus.cc:482] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:43:49.389036 4710 raft_consensus.cc:3037] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:49.391467 4710 raft_consensus.cc:504] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.392012 4710 leader_election.cc:288] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [CANDIDATE]: Term 1 election: Requested vote from peers 8c5490bf8733411f9f590e08eaf22f07 (127.18.186.129:38707), 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605)
I20260808 23:43:49.392534 4358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6d37466f87d4a34863564223afa845b" candidate_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c5490bf8733411f9f590e08eaf22f07"
I20260808 23:43:49.392534 4636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6d37466f87d4a34863564223afa845b" candidate_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09864c13a4e74fc6b3f55b82448b112c"
I20260808 23:43:49.392745 4636 raft_consensus.cc:3037] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:49.392745 4358 raft_consensus.cc:3037] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:43:49.396013 4358 raft_consensus.cc:2454] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c51a5b5485804b7ba7c1736e0b0cd6f9 in term 1.
I20260808 23:43:49.396008 4636 raft_consensus.cc:2454] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c51a5b5485804b7ba7c1736e0b0cd6f9 in term 1.
I20260808 23:43:49.396392 4428 leader_election.cc:302] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09864c13a4e74fc6b3f55b82448b112c, c51a5b5485804b7ba7c1736e0b0cd6f9; no voters:
I20260808 23:43:49.396700 4710 raft_consensus.cc:2781] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:43:49.397300 4710 raft_consensus.cc:686] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 LEADER]: Becoming Leader. State: Replica: c51a5b5485804b7ba7c1736e0b0cd6f9, State: Running, Role: LEADER
I20260808 23:43:49.397886 4710 consensus_queue.cc:237] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } }
I20260808 23:43:49.402738 4202 catalog_manager.cc:5495] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c51a5b5485804b7ba7c1736e0b0cd6f9 (127.18.186.130). New cstate: current_term: 1 leader_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44011 } health_report { overall_health: HEALTHY } } }
I20260808 23:43:49.420964 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.415033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.18.186.130:44011)
0808 23:43:49.415220 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.415226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.415415 (+ 189us) server_negotiation.cc:407] Connection header received
0808 23:43:49.415697 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.415701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.415775 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.415926 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:43:49.416329 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.417060 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.418627 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.419077 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.419220 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.419866 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:43:49.420347 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:43:49.420553 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.420604 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.420686 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:43:49.430160 4636 raft_consensus.cc:1261] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 1 FOLLOWER]: Refusing update from remote peer c51a5b5485804b7ba7c1736e0b0cd6f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:43:49.430160 4358 raft_consensus.cc:1261] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 1 FOLLOWER]: Refusing update from remote peer c51a5b5485804b7ba7c1736e0b0cd6f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:43:49.431296 4714 consensus_queue.cc:1035] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09864c13a4e74fc6b3f55b82448b112c" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:43:49.431742 4710 consensus_queue.cc:1035] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c5490bf8733411f9f590e08eaf22f07" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 38707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:43:49.443222 4717 mvcc.cc:205] Tried to move back new op lower bound from 7316408850135240704 to 7316408850018766848. Current Snapshot: MvccSnapshot[applied={T|T < 7316408850135240704}]
I20260808 23:43:49.465197 4201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:43:49.465590 4201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:43:49.470459 4201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Sending DeleteTablet for 3 replicas of tablet f6d37466f87d4a34863564223afa845b
I20260808 23:43:49.471954 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 4242
I20260808 23:43:49.472023 4614 tablet_service.cc:1514] Processing DeleteTablet for tablet f6d37466f87d4a34863564223afa845b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:49 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:36406
I20260808 23:43:49.472605 4613 tablet_service.cc:1514] Processing DeleteTablet for tablet f6d37466f87d4a34863564223afa845b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:49 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:36406
I20260808 23:43:49.473459 4336 tablet_service.cc:1514] Processing DeleteTablet for tablet f6d37466f87d4a34863564223afa845b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:49 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:60996
I20260808 23:43:49.474366 4478 tablet_service.cc:1514] Processing DeleteTablet for tablet f6d37466f87d4a34863564223afa845b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:43:49 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:59302
I20260808 23:43:49.475669 4731 tablet_replica.cc:330] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: stopping tablet replica
I20260808 23:43:49.476317 4728 tablet_replica.cc:330] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: stopping tablet replica
I20260808 23:43:49.476922 4731 raft_consensus.cc:2229] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:43:49.477480 4728 raft_consensus.cc:2229] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:43:49.478914 4731 raft_consensus.cc:2258] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:49.479489 4728 raft_consensus.cc:2258] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:49.480551 4730 tablet_replica.cc:330] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: stopping tablet replica
I20260808 23:43:49.481019 4730 raft_consensus.cc:2229] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:43:49.481551 4730 raft_consensus.cc:2258] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:43:49.482185 4728 ts_tablet_manager.cc:1905] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:49.482527 4731 ts_tablet_manager.cc:1905] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:49.490276 4731 ts_tablet_manager.cc:1918] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:49.490473 4731 log.cc:1196] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/f6d37466f87d4a34863564223afa845b
I20260808 23:43:49.490689 4730 ts_tablet_manager.cc:1905] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:43:49.491060 4731 ts_tablet_manager.cc:1939] T f6d37466f87d4a34863564223afa845b P c51a5b5485804b7ba7c1736e0b0cd6f9: Deleting consensus metadata
I20260808 23:43:49.493049 4188 catalog_manager.cc:4841] TS c51a5b5485804b7ba7c1736e0b0cd6f9 (127.18.186.130:44011): tablet f6d37466f87d4a34863564223afa845b (table test-table [id=5737875fa2364bad8b8eae50145e9c8a]) successfully deleted
I20260808 23:43:49.496174 4728 ts_tablet_manager.cc:1918] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:49.496443 4728 log.cc:1196] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/f6d37466f87d4a34863564223afa845b
I20260808 23:43:49.497128 4728 ts_tablet_manager.cc:1939] T f6d37466f87d4a34863564223afa845b P 09864c13a4e74fc6b3f55b82448b112c: Deleting consensus metadata
I20260808 23:43:49.498297 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.472993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.18.186.129:38707)
0808 23:43:49.473162 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.473168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.478125 (+ 4957us) server_negotiation.cc:407] Connection header received
0808 23:43:49.478323 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.478329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.478429 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.478532 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:49.481979 (+ 3447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.483092 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.485201 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.485829 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.485936 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.490049 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:49.490070 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:49.490087 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:49.490136 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:49.494213 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:49.495122 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:49.495131 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:49.495135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:49.495319 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:49.495956 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:49.495960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:49.495964 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:49.496542 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:49.496881 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.497603 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.497855 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:43:49.498803 4730 ts_tablet_manager.cc:1918] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:43:49.498970 4730 log.cc:1196] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/f6d37466f87d4a34863564223afa845b
I20260808 23:43:49.499567 4730 ts_tablet_manager.cc:1939] T f6d37466f87d4a34863564223afa845b P 8c5490bf8733411f9f590e08eaf22f07: Deleting consensus metadata
I20260808 23:43:49.499557 4187 catalog_manager.cc:4841] TS 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605): tablet f6d37466f87d4a34863564223afa845b (table test-table [id=5737875fa2364bad8b8eae50145e9c8a]) successfully deleted
I20260808 23:43:49.500015 4402 generic_service.cc:196] Checking for leaks (request via RPC)
W20260808 23:43:49.504952 4187 catalog_manager.cc:4808] TS 09864c13a4e74fc6b3f55b82448b112c (127.18.186.131:36605): delete failed for tablet f6d37466f87d4a34863564223afa845b because the tablet was not found. No further retry: Not found: Tablet not found: f6d37466f87d4a34863564223afa845b
I20260808 23:43:49.591646 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 4242
I20260808 23:43:49.591831 4185 catalog_manager.cc:4841] TS 8c5490bf8733411f9f590e08eaf22f07 (127.18.186.129:38707): tablet f6d37466f87d4a34863564223afa845b (table test-table [id=5737875fa2364bad8b8eae50145e9c8a]) successfully deleted
I20260808 23:43:49.613022 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 4411
I20260808 23:43:49.629609 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.613852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.18.186.130:44011)
0808 23:43:49.614045 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.614050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.614073 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:43:49.614213 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.614223 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.614307 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.614420 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:49.616073 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.617342 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.618652 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.619307 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.619576 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.622645 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:49.622662 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:49.622683 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:49.622732 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:49.626702 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:49.627402 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:49.627412 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:49.627416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:49.627612 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:49.627975 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:49.627980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:49.627984 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:49.628425 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:49.628767 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.628948 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.629189 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:43:49.630308 4540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:49.744163 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 4411
I20260808 23:43:49.765991 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 4548
I20260808 23:43:49.783165 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:43:49.766754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36414 (local address 127.18.186.131:36605)
0808 23:43:49.766960 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:43:49.766965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:43:49.766983 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:43:49.767047 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:43:49.767050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:43:49.767110 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:43:49.767204 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:43:49.768634 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.769371 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.770866 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:43:49.771262 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:43:49.771403 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:43:49.777624 (+ 6221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:43:49.777638 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:43:49.777655 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:43:49.777689 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:43:49.780380 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:49.781058 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:49.781066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:49.781068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:49.781195 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:43:49.781581 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:43:49.781587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:43:49.781590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:43:49.781952 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:43:49.782264 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 23:43:49.782590 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 23:43:49.782810 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:43:49.783906 4678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:49.872066 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 4548
I20260808 23:43:49.891312 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 4168
I20260808 23:43:49.892578 4234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:43:50.006771 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 4168
I20260808 23:44:10.033529 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42427
--webserver_interface=127.18.186.190
--webserver_port=43799
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:10.374970 4736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:10.375320 4736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:10.375530 4736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:10.382763 4736 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:44:10.382899 4736 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:44:10.382995 4736 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:44:10.383080 4736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:10.383179 4736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:10.383278 4736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:10.383366 4736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:10.383466 4736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:10.393435 4736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:10.394745 4736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:10.396171 4736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:10.408829 4742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:10.409050 4744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:10.409176 4741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:10.409473 4736 server_base.cc:1034] running on GCE node
I20260808 23:44:10.410269 4736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:10.411736 4736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:10.412979 4736 hybrid_clock.cc:648] HybridClock initialized: now 1786232650412902 us; error 52 us; skew 500 ppm
I20260808 23:44:10.413312 4736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:10 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:10.416653 4736 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:44:10.418056 4736 webserver.cc:460] Webserver started at http://127.18.186.190:43799/ using document root <none> and password file <none>
I20260808 23:44:10.418749 4736 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:10.418882 4736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:10.424333 4736 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:44:10.427162 4755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:10.428435 4736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:44:10.428620 4736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "36a7abc22b4f458b8504f82c27c07375"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "1a24c49f70df55cf0eca9aada240e2b9"
server_key_iv: "69298c773cd1ade87c61577f093f5ee7"
server_key_version: "encryptionkey@0"
I20260808 23:44:10.429553 4736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:10.462369 4736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:10.466517 4736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:10.466822 4736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:10.486541 4736 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42427
I20260808 23:44:10.486588 4809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42427 every 8 connection(s)
I20260808 23:44:10.488086 4736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:10.492359 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 4736
I20260808 23:44:10.493214 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:38707
--local_ip_for_outbound_sockets=127.18.186.129
--tserver_master_addrs=127.18.186.190:42427
--webserver_port=33801
--webserver_interface=127.18.186.129
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:44:10.495146 4810 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:44:10.498234 4810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Bootstrap starting.
I20260808 23:44:10.519371 4810 log.cc:824] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:10.536783 4810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:44:10.537374 4810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Bootstrap complete.
I20260808 23:44:10.546511 4810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:44:10.547467 4810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a7abc22b4f458b8504f82c27c07375, State: Initialized, Role: FOLLOWER
I20260808 23:44:10.548138 4810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:44:10.548388 4810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:10.548518 4810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:10.548709 4810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 2 FOLLOWER]: Advancing to term 3
I20260808 23:44:10.552481 4810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:44:10.552958 4810 leader_election.cc:302] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a7abc22b4f458b8504f82c27c07375; no voters:
I20260808 23:44:10.553678 4810 leader_election.cc:288] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 23:44:10.553817 4814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 23:44:10.555236 4814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [term 3 LEADER]: Becoming Leader. State: Replica: 36a7abc22b4f458b8504f82c27c07375, State: Running, Role: LEADER
I20260808 23:44:10.555873 4814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } }
I20260808 23:44:10.556100 4810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:10.558400 4816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a7abc22b4f458b8504f82c27c07375. Latest consensus state: current_term: 3 leader_uuid: "36a7abc22b4f458b8504f82c27c07375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } } }
I20260808 23:44:10.558449 4815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "36a7abc22b4f458b8504f82c27c07375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a7abc22b4f458b8504f82c27c07375" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42427 } } }
I20260808 23:44:10.558802 4816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:10.558876 4815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:10.565340 4820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:10.574193 4820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:10.575721 4820 catalog_manager.cc:1259] Loaded cluster ID: bbe6cd9a9b86495abee67736a9ddab7b
I20260808 23:44:10.575886 4820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:10.579908 4820 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:10.581540 4820 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Loaded TSK: 0
I20260808 23:44:10.581751 4820 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36a7abc22b4f458b8504f82c27c07375: Loaded TSK: 1
F20260808 23:44:10.582896 4820 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fef6e3389ad google::LogMessage::Fail() at ??:0
@ 0x7fef6e33b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fef6e3384de google::LogMessage::Flush() at ??:0
@ 0x7fef6e339a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fef7209c7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fef7208b73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fef83d671c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fef83d66783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
I20260808 23:44:10.623308 4836 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fef83df22a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fef83df20c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fef83e3c5e9 std::function<>::operator()() at ??:0
@ 0x7fef83d6acdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fef83d68c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fef83df0610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fef83df0439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fef70f84345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fef70f8791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fef70f87799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fef70f62eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fef7423c6db start_thread at ??:0
@ 0x7fef6d1d261f clone at ??:0
W20260808 23:44:10.886904 4812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:10.887184 4812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:10.887282 4812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:10.894588 4812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:10.894693 4812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:10.894811 4812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:44:10.904603 4812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:38707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=33801
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:10.905848 4812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:10.907123 4812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:10.920735 4864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:10.921141 4861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:10.921196 4862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:10.921788 4812 server_base.cc:1034] running on GCE node
I20260808 23:44:10.922401 4812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:10.923343 4812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:10.924542 4812 hybrid_clock.cc:648] HybridClock initialized: now 1786232650924488 us; error 47 us; skew 500 ppm
I20260808 23:44:10.924875 4812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:10 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:10.928001 4812 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:44:10.929390 4812 webserver.cc:460] Webserver started at http://127.18.186.129:33801/ using document root <none> and password file <none>
I20260808 23:44:10.930078 4812 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:10.930226 4812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:10.935662 4812 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:44:10.938262 4871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:10.939616 4812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:10.939792 4812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "8c5490bf8733411f9f590e08eaf22f07"
format_stamp: "Formatted at 2026-08-08 23:43:24 on dist-test-slave-5k54"
server_key: "d81a81a3d74d446018ae7e97e4a23f45"
server_key_iv: "35f5209dbb8bb0dfdc2a5783cb702c54"
server_key_version: "encryptionkey@0"
I20260808 23:44:10.940749 4812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:10.959594 4812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:10.963457 4812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:10.963732 4812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:10.964695 4812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:10.966078 4812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:10.966202 4812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:10.966343 4812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:10.966436 4812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:11.001014 4812 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:38707
I20260808 23:44:11.001101 4988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:38707 every 8 connection(s)
I20260808 23:44:11.002471 4812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:44:11.003167 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 4812
I20260808 23:44:11.004045 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:44011
--local_ip_for_outbound_sockets=127.18.186.130
--tserver_master_addrs=127.18.186.190:42427
--webserver_port=39021
--webserver_interface=127.18.186.130
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:44:11.008184 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.006094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:33563)
0808 23:44:11.006905 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.007231 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":348,"threads_started":1}
W20260808 23:44:11.008963 4989 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:42427 (0 consecutive failures): Network error: Failed to ping master at 127.18.186.190:42427: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
I20260808 23:44:11.009786 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.009458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:49717)
0808 23:44:11.009604 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.009686 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 23:44:11.010679 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.010396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:52827)
0808 23:44:11.010514 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.010582 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 23:44:11.010972 4989 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:44:11.342511 4992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:11.342789 4992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:11.342985 4992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:11.350476 4992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:11.350584 4992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:11.350787 4992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:44:11.360414 4992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:44011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=39021
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:11.361660 4992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:11.362977 4992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:11.376366 4997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:11.376322 5000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:11.376397 4998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:11.377100 4992 server_base.cc:1034] running on GCE node
I20260808 23:44:11.377637 4992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:11.378491 4992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:11.379681 4992 hybrid_clock.cc:648] HybridClock initialized: now 1786232651379624 us; error 50 us; skew 500 ppm
I20260808 23:44:11.379957 4992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:11 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232651, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:11.383116 4992 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:44:11.384642 4992 webserver.cc:460] Webserver started at http://127.18.186.130:39021/ using document root <none> and password file <none>
I20260808 23:44:11.385330 4992 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:11.385476 4992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:11.390631 4992 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:44:11.395093 5007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:11.396618 4992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:44:11.396804 4992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "c51a5b5485804b7ba7c1736e0b0cd6f9"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "a66f85187830733e639782abfb301f68"
server_key_iv: "253d18886a5891a886b90ff147894090"
server_key_version: "encryptionkey@0"
I20260808 23:44:11.397804 4992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:11.419898 4992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:11.423781 4992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:11.424053 4992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:11.425192 4992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:11.426527 4992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:11.426695 4992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:11.426823 4992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:11.426913 4992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:11.461297 4992 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:44011
I20260808 23:44:11.461390 5127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:44011 every 8 connection(s)
I20260808 23:44:11.462780 4992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:44:11.464073 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 4992
I20260808 23:44:11.464869 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:36605
--local_ip_for_outbound_sockets=127.18.186.131
--tserver_master_addrs=127.18.186.190:42427
--webserver_port=33565
--webserver_interface=127.18.186.131
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:44:11.468411 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.466413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:51857)
0808 23:44:11.467230 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.467576 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":328,"threads_started":1}
W20260808 23:44:11.469076 5128 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:42427 (0 consecutive failures): Network error: Failed to ping master at 127.18.186.190:42427: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
I20260808 23:44:11.469969 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.469643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:45059)
0808 23:44:11.469787 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.469856 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 23:44:11.470858 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.470564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:35989)
0808 23:44:11.470685 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.470754 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 23:44:11.471138 5128 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:44:11.799817 5131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:11.800086 5131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:11.800179 5131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:11.807476 5131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:11.807590 5131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:11.807706 5131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:44:11.817369 5131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:36605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=33565
--tserver_master_addrs=127.18.186.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:11.818579 5131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:11.819875 5131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:11.833534 5139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:11.833442 5137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:11.833592 5136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:11.834630 5131 server_base.cc:1034] running on GCE node
I20260808 23:44:11.835243 5131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:11.836210 5131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:11.837437 5131 hybrid_clock.cc:648] HybridClock initialized: now 1786232651837380 us; error 47 us; skew 500 ppm
I20260808 23:44:11.837754 5131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:11 dist-test-slave-5k54 krb5kdc[3571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232651, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:11.841009 5131 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:44:11.842419 5131 webserver.cc:460] Webserver started at http://127.18.186.131:33565/ using document root <none> and password file <none>
I20260808 23:44:11.843060 5131 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:11.843159 5131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:11.848683 5131 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:44:11.851301 5147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:11.852641 5131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:11.852818 5131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "09864c13a4e74fc6b3f55b82448b112c"
format_stamp: "Formatted at 2026-08-08 23:43:25 on dist-test-slave-5k54"
server_key: "1648097478eb6549e0b73305d9307ebd"
server_key_iv: "6f03ef7f05ce1881605ec4c91b2235f5"
server_key_version: "encryptionkey@0"
I20260808 23:44:11.853771 5131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:11.878788 5131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:11.883144 5131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:11.883479 5131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:11.884541 5131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:11.885892 5131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:11.885993 5131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:11.886152 5131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:11.886250 5131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:11.919867 5131 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:36605
I20260808 23:44:11.919942 5271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:36605 every 8 connection(s)
I20260808 23:44:11.921324 5131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:44:11.925433 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 5131
I20260808 23:44:11.926440 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.924233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:38631)
0808 23:44:11.924979 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.925345 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":267,"threads_started":1}
W20260808 23:44:11.927232 5272 heartbeater.cc:644] Failed to heartbeat to 127.18.186.190:42427 (0 consecutive failures): Network error: Failed to ping master at 127.18.186.190:42427: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
I20260808 23:44:11.928038 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.927743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:60275)
0808 23:44:11.927871 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.927936 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 23:44:11.928814 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:11.928501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:59879)
0808 23:44:11.928638 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:11.928717 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:44:11.929042 5272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:44:12.012372 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:12.011581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:55587)
0808 23:44:12.012043 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:12.012145 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":223,"threads_started":1}
I20260808 23:44:12.472429 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:12.471676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:36463)
0808 23:44:12.472115 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:12.472201 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":222,"threads_started":1}
I20260808 23:44:12.930382 5279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:12.929548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:51521)
0808 23:44:12.930069 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:12.930157 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":289,"threads_started":1}
I20260808 23:44:13.014016 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:13.013318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:42199)
0808 23:44:13.013756 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:13.013836 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":220,"threads_started":1}
I20260808 23:44:13.474066 5281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:13.473349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:46299)
0808 23:44:13.473802 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:13.473883 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":227,"threads_started":1}
I20260808 23:44:13.932108 5282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:13.931296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:36785)
0808 23:44:13.931860 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:13.931944 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":282,"threads_started":1}
I20260808 23:44:14.015687 5284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:14.014881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:53139)
0808 23:44:14.015403 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:14.015512 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":274,"threads_started":1}
I20260808 23:44:14.475762 5285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:14.474894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:35397)
0808 23:44:14.475499 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:14.475588 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":301,"threads_started":1}
I20260808 23:44:14.933871 5286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:14.933028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:45429)
0808 23:44:14.933542 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:14.933650 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":266,"threads_started":1}
I20260808 23:44:15.017459 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:15.016676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:49093)
0808 23:44:15.017183 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:15.017268 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":278,"threads_started":1}
I20260808 23:44:15.477267 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:15.476585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:48059)
0808 23:44:15.477004 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:15.477081 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":214,"threads_started":1}
I20260808 23:44:15.935624 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:15.934761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:34407)
0808 23:44:15.935295 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:15.935373 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":281,"threads_started":1}
I20260808 23:44:16.019095 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:16.018373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:47893)
0808 23:44:16.018836 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:16.018922 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":237,"threads_started":1}
I20260808 23:44:16.478911 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:16.478208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:37859)
0808 23:44:16.478651 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:16.478730 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":228,"threads_started":1}
I20260808 23:44:16.937171 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:16.936469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:39631)
0808 23:44:16.936938 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:16.937015 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":256,"threads_started":1}
I20260808 23:44:17.020921 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:17.020034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:38157)
0808 23:44:17.020583 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:17.020683 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":259,"threads_started":1}
I20260808 23:44:17.480679 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:17.479847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:50693)
0808 23:44:17.480352 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:17.480477 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":253,"threads_started":1}
I20260808 23:44:17.938958 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:17.938167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:33601)
0808 23:44:17.938702 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:17.938790 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":305,"threads_started":1}
I20260808 23:44:18.022749 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:18.021936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:44131)
0808 23:44:18.022426 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:18.022528 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":210,"threads_started":1}
I20260808 23:44:18.482474 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:18.481646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:33719)
0808 23:44:18.482173 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:18.482256 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":256,"threads_started":1}
I20260808 23:44:18.940783 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:18.939965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:58977)
0808 23:44:18.940475 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:18.940556 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":251,"threads_started":1}
I20260808 23:44:19.024387 5299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:19.023714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:56705)
0808 23:44:19.024140 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:19.024221 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":215,"threads_started":1}
I20260808 23:44:19.484037 5300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:19.483344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:51099)
0808 23:44:19.483794 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:19.483876 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":240,"threads_started":1}
I20260808 23:44:19.942330 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:19.941629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:52905)
0808 23:44:19.942085 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:19.942168 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":227,"threads_started":1}
I20260808 23:44:20.026080 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:20.025243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:35519)
0808 23:44:20.025777 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:20.025869 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":276,"threads_started":1}
I20260808 23:44:20.485710 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:20.484954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:60353)
0808 23:44:20.485449 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:20.485533 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":230,"threads_started":1}
I20260808 23:44:20.943944 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:20.943143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:58997)
0808 23:44:20.943651 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:20.943726 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":227,"threads_started":1}
I20260808 23:44:21.027786 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:21.027093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:37767)
0808 23:44:21.027543 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:21.027626 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":233,"threads_started":1}
I20260808 23:44:21.487358 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:21.486630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:48039)
0808 23:44:21.487106 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:21.487192 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":257,"threads_started":1}
I20260808 23:44:21.945510 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:21.944750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:48491)
0808 23:44:21.945199 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:21.945280 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":235,"threads_started":1}
I20260808 23:44:22.029501 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:22.028719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:43257)
0808 23:44:22.029205 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:22.029304 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":254,"threads_started":1}
I20260808 23:44:22.488997 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:22.488264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:58031)
0808 23:44:22.488718 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:22.488812 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":212,"threads_started":1}
I20260808 23:44:22.947331 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:22.946352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:36985)
0808 23:44:22.946906 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:22.947047 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":278,"threads_started":1}
I20260808 23:44:23.031122 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:23.030421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:58089)
0808 23:44:23.030883 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:23.030967 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":224,"threads_started":1}
I20260808 23:44:23.490613 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:23.489883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:47999)
0808 23:44:23.490357 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:23.490459 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":255,"threads_started":1}
I20260808 23:44:23.948854 5314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:23.948215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:33373)
0808 23:44:23.948623 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:23.948703 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":204,"threads_started":1}
I20260808 23:44:24.032910 5315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:24.032020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:41989)
0808 23:44:24.032574 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:24.032675 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":296,"threads_started":1}
I20260808 23:44:24.492420 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:24.491509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:36781)
0808 23:44:24.492037 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:24.492168 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":253,"threads_started":1}
I20260808 23:44:24.950482 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:24.949707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:56651)
0808 23:44:24.950196 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:24.950289 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":227,"threads_started":1}
I20260808 23:44:25.034590 5319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:25.033857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:38763)
0808 23:44:25.034297 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:25.034389 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":228,"threads_started":1}
I20260808 23:44:25.494196 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:25.493358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:50067)
0808 23:44:25.493875 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:25.493996 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":248,"threads_started":1}
I20260808 23:44:25.952031 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:25.951301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:54009)
0808 23:44:25.951763 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:25.951853 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":253,"threads_started":1}
I20260808 23:44:26.036502 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:26.035719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:55777)
0808 23:44:26.036208 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:26.036308 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":272,"threads_started":1}
I20260808 23:44:26.496060 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:26.495220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.130:55319)
0808 23:44:26.495752 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:26.495880 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":318,"threads_started":1}
I20260808 23:44:26.934091 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 4812
I20260808 23:44:26.955561 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:26.953196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.131:36513)
0808 23:44:26.955132 (+ 1936us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:26.955281 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1791,"thread_start_us":1668,"threads_started":1}
I20260808 23:44:26.958098 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:26.935242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37638 (local address 127.18.186.129:38707)
0808 23:44:26.936010 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:26.936027 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:26.936092 (+ 65us) server_negotiation.cc:407] Connection header received
0808 23:44:26.938040 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:26.938050 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:26.938494 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:26.938866 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:26.940401 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:26.941658 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:26.942952 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:26.943791 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:26.943989 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:26.951362 (+ 7373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:26.951371 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:26.951387 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:26.951469 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:26.954605 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:26.955519 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:26.955526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:26.955533 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:26.955766 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:26.956196 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:26.956199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:26.956201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:26.956730 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:26.957010 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:26.957453 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 23:44:26.957790 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":379,"threads_started":1}
I20260808 23:44:26.959628 4984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:27.048300 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:27.043298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42427 (local address 127.18.186.129:45817)
0808 23:44:27.047854 (+ 4556us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:27.048032 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.190:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4423,"thread_start_us":4274,"threads_started":1}
I20260808 23:44:27.049134 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 4812
I20260808 23:44:27.069181 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 4992
I20260808 23:44:27.087301 5331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:27.070244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.18.186.130:44011)
0808 23:44:27.070792 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:27.070806 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:27.070847 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:44:27.072524 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:27.072534 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:27.072970 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:27.073257 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:27.074658 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:27.076497 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:27.077686 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:27.078373 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:27.078514 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:27.081327 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:27.081334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:27.081348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:27.081384 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:27.084385 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:27.085085 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:27.085092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:27.085098 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:27.085263 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:27.085576 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:27.085579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:27.085581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:27.086074 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:27.086295 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:27.086618 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 23:44:27.087026 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":268,"threads_started":1}
I20260808 23:44:27.088738 5123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:27.172950 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 4992
I20260808 23:44:27.193670 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 5131
I20260808 23:44:27.211164 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:27.194730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39802 (local address 127.18.186.131:36605)
0808 23:44:27.195347 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:27.195362 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:27.195411 (+ 49us) server_negotiation.cc:407] Connection header received
0808 23:44:27.197116 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:27.197125 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:27.197549 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:27.197825 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:27.199180 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:27.200452 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:27.201677 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:27.202294 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:27.202429 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:27.205143 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:27.205151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:27.205164 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:27.205204 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:27.208289 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:27.208939 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:27.208945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:27.208951 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:27.209105 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:27.209412 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:27.209415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:27.209417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:27.209895 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:27.210107 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:27.210473 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 23:44:27.210867 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":298,"threads_started":1}
I20260808 23:44:27.212592 5267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:27.296805 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 5131
2026-08-08T23:44:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64827 ms)
[----------] 30 tests from SecurityITest (300549 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:44:27 dist-test-slave-5k54 krb5kdc[5338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:44:27 dist-test-slave-5k54 krb5kdc[5338](info): set up 2 sockets
Aug 08 23:44:27 dist-test-slave-5k54 krb5kdc[5338](info): commencing operation
krb5kdc: starting...
W20260808 23:44:28.497809 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:44:28 dist-test-slave-5k54 krb5kdc[5338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:44:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:44:28.782899 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:41849
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:41849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:29.127897 5353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:29.128232 5353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:29.128402 5353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:29.135769 5353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:29.135914 5353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:29.136023 5353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:29.136127 5353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:29.136202 5353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:29.146720 5353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:41849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:41849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:29.148159 5353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:29.149494 5353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:29.161468 5363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:29.161445 5359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:29.161899 5353 server_base.cc:1034] running on GCE node
W20260808 23:44:29.161518 5360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:29.162860 5353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:29.164310 5353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:29.165542 5353 hybrid_clock.cc:648] HybridClock initialized: now 1786232669165502 us; error 43 us; skew 500 ppm
I20260808 23:44:29.165880 5353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:29 dist-test-slave-5k54 krb5kdc[5338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232669, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:29.168987 5353 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:44:29.170492 5353 webserver.cc:460] Webserver started at http://127.18.186.190:37663/ using document root <none> and password file <none>
I20260808 23:44:29.171211 5353 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:29.171362 5353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:29.171695 5353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:29.174612 5353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "fa2da32defaf4672821dc5a1c638ed20"
format_stamp: "Formatted at 2026-08-08 23:44:29 on dist-test-slave-5k54"
server_key: "2a0866cd104c1cb59f96205cc33c47b4"
server_key_iv: "00ca86a8ff34eb343b480756f74bcf35"
server_key_version: "encryptionkey@0"
I20260808 23:44:29.175393 5353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "fa2da32defaf4672821dc5a1c638ed20"
format_stamp: "Formatted at 2026-08-08 23:44:29 on dist-test-slave-5k54"
server_key: "2a0866cd104c1cb59f96205cc33c47b4"
server_key_iv: "00ca86a8ff34eb343b480756f74bcf35"
server_key_version: "encryptionkey@0"
I20260808 23:44:29.180487 5353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 23:44:29.183974 5371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:29.185307 5353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:44:29.185489 5353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "fa2da32defaf4672821dc5a1c638ed20"
format_stamp: "Formatted at 2026-08-08 23:44:29 on dist-test-slave-5k54"
server_key: "2a0866cd104c1cb59f96205cc33c47b4"
server_key_iv: "00ca86a8ff34eb343b480756f74bcf35"
server_key_version: "encryptionkey@0"
I20260808 23:44:29.185704 5353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:29.206324 5353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:29.210613 5353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:29.210903 5353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:29.229856 5353 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:41849
I20260808 23:44:29.229876 5423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:41849 every 8 connection(s)
I20260808 23:44:29.231256 5353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:29.232178 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 5353
I20260808 23:44:29.232416 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:29.233135 19178 external_mini_cluster.cc:1428] Setting key 00224ce73a66369fb5bc0a76e9166d9e
I20260808 23:44:29.235729 5424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:29.246292 5424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20: Bootstrap starting.
Aug 08 23:44:29 dist-test-slave-5k54 krb5kdc[5338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:29.249533 5424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:29.250710 5424 log.cc:824] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:29.253969 5424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20: No bootstrap required, opened a new log
I20260808 23:44:29.255743 5428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:29.235175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.18.186.190:41849)
0808 23:44:29.236101 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:29.236113 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:29.236154 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:44:29.237845 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:29.237866 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:29.238272 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:29.238687 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:29.240276 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:29.241582 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:29.242933 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:29.243702 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:29.243924 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:29.248176 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:29.248184 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:29.248197 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:29.248245 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:29.251569 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:29.252366 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:29.252372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:29.252378 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:29.252545 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:29.253013 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:29.253016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:29.253018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:29.253514 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:29.253728 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:29.254270 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 23:44:29.254645 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":438,"threads_started":1}
I20260808 23:44:29.262297 5424 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2da32defaf4672821dc5a1c638ed20" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41849 } }
I20260808 23:44:29.262660 5424 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:29.262780 5424 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa2da32defaf4672821dc5a1c638ed20, State: Initialized, Role: FOLLOWER
I20260808 23:44:29.263321 5424 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2da32defaf4672821dc5a1c638ed20" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41849 } }
I20260808 23:44:29.263598 5424 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:29.263764 5424 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:29.263926 5424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:29.266328 5424 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2da32defaf4672821dc5a1c638ed20" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41849 } }
I20260808 23:44:29.266849 5424 leader_election.cc:302] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa2da32defaf4672821dc5a1c638ed20; no voters:
I20260808 23:44:29.267513 5424 leader_election.cc:288] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:29.267678 5430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:29.268383 5430 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [term 1 LEADER]: Becoming Leader. State: Replica: fa2da32defaf4672821dc5a1c638ed20, State: Running, Role: LEADER
I20260808 23:44:29.268898 5430 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2da32defaf4672821dc5a1c638ed20" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41849 } }
I20260808 23:44:29.269580 5424 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:29.271773 5432 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa2da32defaf4672821dc5a1c638ed20. Latest consensus state: current_term: 1 leader_uuid: "fa2da32defaf4672821dc5a1c638ed20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2da32defaf4672821dc5a1c638ed20" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41849 } } }
I20260808 23:44:29.272117 5432 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:29.273058 5431 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa2da32defaf4672821dc5a1c638ed20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2da32defaf4672821dc5a1c638ed20" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41849 } } }
I20260808 23:44:29.273478 5431 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:29.274283 5438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:29.279578 5438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:29.288836 5438 catalog_manager.cc:1347] Generated new cluster ID: 399b4309727d4459891927b343c8152c
I20260808 23:44:29.288977 5438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:29.303016 5438 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:29.304306 5438 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:29.315181 5438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20: Generated new TSK 0
I20260808 23:44:29.316238 5438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:44:29.395702 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41849
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:29.742779 5451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:29.743053 5451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:29.743206 5451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:29.750461 5451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:29.750564 5451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:29.750767 5451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:44:29.760764 5451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:29.762027 5451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:29.763332 5451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:29.776892 5460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:29.777012 5458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:29.776855 5457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:29.777659 5451 server_base.cc:1034] running on GCE node
I20260808 23:44:29.778281 5451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:29.779193 5451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:29.780428 5451 hybrid_clock.cc:648] HybridClock initialized: now 1786232669780358 us; error 78 us; skew 500 ppm
I20260808 23:44:29.780745 5451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:29 dist-test-slave-5k54 krb5kdc[5338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232669, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:29.783951 5451 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:44:29.785733 5451 webserver.cc:460] Webserver started at http://127.18.186.129:37233/ using document root <none> and password file <none>
I20260808 23:44:29.786404 5451 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:29.786550 5451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:29.786862 5451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:29.789762 5451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "9e8971b1698b462ea5bf6ed8d588dc25"
format_stamp: "Formatted at 2026-08-08 23:44:29 on dist-test-slave-5k54"
server_key: "de9c32d9516df7065e59eda13e9bf58d"
server_key_iv: "14c605b45a7a928187025870685f9bd0"
server_key_version: "encryptionkey@0"
I20260808 23:44:29.790520 5451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "9e8971b1698b462ea5bf6ed8d588dc25"
format_stamp: "Formatted at 2026-08-08 23:44:29 on dist-test-slave-5k54"
server_key: "de9c32d9516df7065e59eda13e9bf58d"
server_key_iv: "14c605b45a7a928187025870685f9bd0"
server_key_version: "encryptionkey@0"
I20260808 23:44:29.795544 5451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260808 23:44:29.799058 5467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:29.800464 5451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:44:29.800638 5451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "9e8971b1698b462ea5bf6ed8d588dc25"
format_stamp: "Formatted at 2026-08-08 23:44:29 on dist-test-slave-5k54"
server_key: "de9c32d9516df7065e59eda13e9bf58d"
server_key_iv: "14c605b45a7a928187025870685f9bd0"
server_key_version: "encryptionkey@0"
I20260808 23:44:29.800812 5451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:29.817232 5451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:29.821525 5451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:29.821815 5451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:29.822809 5451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:29.824249 5451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:29.824324 5451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:29.824477 5451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:29.824540 5451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:29.858812 5451 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:46049
I20260808 23:44:29.858856 5590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:46049 every 8 connection(s)
I20260808 23:44:29.860167 5451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:44:29.867065 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 5451
I20260808 23:44:29.867321 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:44:29.868083 19178 external_mini_cluster.cc:1428] Setting key f4b618f37b47dd2c7473c78b14b1dfa7
Aug 08 23:44:29 dist-test-slave-5k54 krb5kdc[5338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232669, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:29.883641 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:29.862867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:57971 (local address 127.18.186.190:41849)
0808 23:44:29.863370 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:29.863374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:29.864344 (+ 970us) server_negotiation.cc:407] Connection header received
0808 23:44:29.866233 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:29.866239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:29.866335 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:29.866458 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:29.868178 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:29.869217 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:29.871482 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:29.871951 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:29.872062 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:29.876332 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:29.876341 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:29.876349 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:29.876393 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:29.880191 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:29.881007 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:29.881016 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:29.881022 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:29.881179 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:29.881615 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:29.881622 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:29.881628 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:29.882014 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:29.882174 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:29.882911 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 23:44:29.883178 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":280,"threads_started":1}
I20260808 23:44:29.884783 5594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:29.863345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41849 (local address 127.18.186.129:57971)
0808 23:44:29.864175 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:29.864214 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:29.865959 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:29.866714 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:29.866722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:29.867337 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:29.867994 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:29.868012 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:29.869403 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:29.869407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:29.871263 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:29.871282 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:29.872195 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:29.872206 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:29.872558 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:29.873302 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:29.873336 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:29.876089 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:29.880363 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:29.880371 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:29.880384 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:29.880866 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:29.881293 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:29.881296 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:29.881299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:29.881495 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:29.882154 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:29.882160 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:29.882495 (+ 335us) client_negotiation.cc:769] Sending connection context
0808 23:44:29.882742 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 23:44:29.883180 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":320,"threads_started":1}
I20260808 23:44:29.886511 5591 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41849
I20260808 23:44:29.886821 5591 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:29.887612 5591 heartbeater.cc:511] Master 127.18.186.190:41849 requested a full tablet report, sending...
I20260808 23:44:29.889801 5389 ts_manager.cc:194] Registered new tserver with Master: 9e8971b1698b462ea5bf6ed8d588dc25 (127.18.186.129:46049)
I20260808 23:44:29.891350 5389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:57971
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:44:29.938381 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41849
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:30.281132 5597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:30.281446 5597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:30.281622 5597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:30.288969 5597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:30.289119 5597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:30.289306 5597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:44:30.299178 5597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:30.300449 5597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:30.301699 5597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:30.314872 5603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:30.314934 5602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:30.315026 5605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:30.316123 5597 server_base.cc:1034] running on GCE node
I20260808 23:44:30.316707 5597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:30.317603 5597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:30.318831 5597 hybrid_clock.cc:648] HybridClock initialized: now 1786232670318787 us; error 57 us; skew 500 ppm
I20260808 23:44:30.319154 5597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:30 dist-test-slave-5k54 krb5kdc[5338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232670, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:30.322297 5597 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:44:30.323774 5597 webserver.cc:460] Webserver started at http://127.18.186.130:45197/ using document root <none> and password file <none>
I20260808 23:44:30.324433 5597 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:30.324584 5597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:30.324903 5597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:30.327800 5597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "cc14ef33f28e4a758b0b609c7b420c5e"
format_stamp: "Formatted at 2026-08-08 23:44:30 on dist-test-slave-5k54"
server_key: "d2c5d59e20dafed37c58db444b327655"
server_key_iv: "922a86cfadb74a3b54784b38ee1d5f15"
server_key_version: "encryptionkey@0"
I20260808 23:44:30.328569 5597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "cc14ef33f28e4a758b0b609c7b420c5e"
format_stamp: "Formatted at 2026-08-08 23:44:30 on dist-test-slave-5k54"
server_key: "d2c5d59e20dafed37c58db444b327655"
server_key_iv: "922a86cfadb74a3b54784b38ee1d5f15"
server_key_version: "encryptionkey@0"
I20260808 23:44:30.333604 5597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:44:30.336990 5613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:30.338359 5597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:44:30.338541 5597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "cc14ef33f28e4a758b0b609c7b420c5e"
format_stamp: "Formatted at 2026-08-08 23:44:30 on dist-test-slave-5k54"
server_key: "d2c5d59e20dafed37c58db444b327655"
server_key_iv: "922a86cfadb74a3b54784b38ee1d5f15"
server_key_version: "encryptionkey@0"
I20260808 23:44:30.338747 5597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:30.363090 5597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:30.367058 5597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:30.367348 5597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:30.368330 5597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:30.369674 5597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:30.369778 5597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:30.369925 5597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:30.370005 5597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:30.403044 5597 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:42011
I20260808 23:44:30.403097 5732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:42011 every 8 connection(s)
I20260808 23:44:30.404489 5597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:44:30.407377 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 5597
I20260808 23:44:30.407618 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:44:30.408104 19178 external_mini_cluster.cc:1428] Setting key f8efffb40af0d4f95672f16e61185c7f
Aug 08 23:44:30 dist-test-slave-5k54 krb5kdc[5338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232670, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:30.425891 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:30.407196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:43275 (local address 127.18.186.190:41849)
0808 23:44:30.407700 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:30.407708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:30.408781 (+ 1073us) server_negotiation.cc:407] Connection header received
0808 23:44:30.410928 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:30.410935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:30.411024 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:30.411144 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:30.412752 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:30.413895 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:30.415395 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:30.416151 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:30.416292 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:30.420088 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:30.420092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:30.420096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:30.420127 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:30.422680 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:30.423645 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:30.423649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:30.423652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:30.423777 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:30.424253 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:30.424256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:30.424258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:30.424511 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:30.424622 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:30.425461 (+ 839us) server_negotiation.cc:299] Negotiation successful
0808 23:44:30.425624 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":252,"threads_started":1}
I20260808 23:44:30.427354 5736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:30.407780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41849 (local address 127.18.186.130:43275)
0808 23:44:30.408597 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:30.408642 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:30.410656 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:30.411375 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:30.411387 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:30.411961 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:30.412586 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:30.412604 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:30.414043 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:30.414046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:30.415241 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:30.415253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:30.416344 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:30.416349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:30.416534 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:30.417197 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:30.417218 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:30.419853 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:30.422816 (+ 2963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:30.422827 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:30.422842 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:30.423518 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:30.423891 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:30.423899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:30.423906 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:30.424143 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:30.424640 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:30.424650 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:30.425083 (+ 433us) client_negotiation.cc:769] Sending connection context
0808 23:44:30.425381 (+ 298us) client_negotiation.cc:240] Negotiation successful
0808 23:44:30.425916 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":340,"threads_started":1}
I20260808 23:44:30.428820 5733 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41849
I20260808 23:44:30.429153 5733 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:30.429860 5733 heartbeater.cc:511] Master 127.18.186.190:41849 requested a full tablet report, sending...
I20260808 23:44:30.431294 5389 ts_manager.cc:194] Registered new tserver with Master: cc14ef33f28e4a758b0b609c7b420c5e (127.18.186.130:42011)
I20260808 23:44:30.432149 5389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:43275
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:44:30.475633 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41849
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:30.814240 5739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:30.814560 5739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:30.814731 5739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:30.822017 5739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:30.822172 5739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:30.822358 5739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:44:30.832054 5739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:30.833379 5739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:30.834714 5739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:30.848189 5746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:30.848145 5748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:30.848219 5745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:30.848930 5739 server_base.cc:1034] running on GCE node
I20260808 23:44:30.849526 5739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:30.850493 5739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:30.851711 5739 hybrid_clock.cc:648] HybridClock initialized: now 1786232670851658 us; error 50 us; skew 500 ppm
I20260808 23:44:30.852021 5739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:30 dist-test-slave-5k54 krb5kdc[5338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232670, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:30.855051 5739 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:44:30.856590 5739 webserver.cc:460] Webserver started at http://127.18.186.131:46079/ using document root <none> and password file <none>
I20260808 23:44:30.857262 5739 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:30.857419 5739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:30.857738 5739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:30.860773 5739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "5c707bc7e04047a2ac087f53d03955f5"
format_stamp: "Formatted at 2026-08-08 23:44:30 on dist-test-slave-5k54"
server_key: "778e8c26f2d434c330d074388d590197"
server_key_iv: "0a146f2b2142f3f2afd0188b6e15ac3b"
server_key_version: "encryptionkey@0"
I20260808 23:44:30.861544 5739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "5c707bc7e04047a2ac087f53d03955f5"
format_stamp: "Formatted at 2026-08-08 23:44:30 on dist-test-slave-5k54"
server_key: "778e8c26f2d434c330d074388d590197"
server_key_iv: "0a146f2b2142f3f2afd0188b6e15ac3b"
server_key_version: "encryptionkey@0"
I20260808 23:44:30.866626 5739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:44:30.870407 5756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:30.872291 5739 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 23:44:30.872510 5739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "5c707bc7e04047a2ac087f53d03955f5"
format_stamp: "Formatted at 2026-08-08 23:44:30 on dist-test-slave-5k54"
server_key: "778e8c26f2d434c330d074388d590197"
server_key_iv: "0a146f2b2142f3f2afd0188b6e15ac3b"
server_key_version: "encryptionkey@0"
I20260808 23:44:30.872745 5739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:30.894290 5591 heartbeater.cc:503] Master 127.18.186.190:41849 was elected leader, sending a full tablet report...
I20260808 23:44:30.921885 5739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:30.925818 5739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:30.926101 5739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:30.927145 5739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:30.928587 5739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:30.928694 5739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:30.928839 5739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:30.928908 5739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:30.962442 5739 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:41739
I20260808 23:44:30.962491 5875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:41739 every 8 connection(s)
I20260808 23:44:30.963771 5739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:44:30.965261 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 5739
I20260808 23:44:30.965476 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:44:30.966059 19178 external_mini_cluster.cc:1428] Setting key 5da4a60cd8fe1ee91afa5e12a7732bbd
Aug 08 23:44:30 dist-test-slave-5k54 krb5kdc[5338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232670, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:30.985282 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:30.967232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:45357 (local address 127.18.186.190:41849)
0808 23:44:30.967780 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:30.967788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:30.969034 (+ 1246us) server_negotiation.cc:407] Connection header received
0808 23:44:30.971147 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:30.971151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:30.971230 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:30.971321 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:30.973012 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:30.973800 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:30.975304 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:30.975731 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:30.975820 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:30.979535 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:30.979540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:30.979543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:30.979572 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:30.982533 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:30.983233 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:30.983237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:30.983239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:30.983350 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:30.983771 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:30.983775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:30.983777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:30.984039 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:30.984163 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:30.984847 (+ 684us) server_negotiation.cc:299] Negotiation successful
0808 23:44:30.985012 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":269,"threads_started":1}
I20260808 23:44:30.986259 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:30.967898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41849 (local address 127.18.186.131:45357)
0808 23:44:30.968798 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:30.968909 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:30.970770 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:30.971589 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:30.971598 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:30.972162 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:30.972845 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:30.972871 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:30.973945 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:30.973949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:30.975185 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:30.975197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:30.975924 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:30.975929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:30.976175 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:30.976961 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:30.976981 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:30.979311 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:30.982652 (+ 3341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:30.982658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:30.982670 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:30.983132 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:30.983487 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:30.983490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:30.983493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:30.983676 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:30.984191 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:30.984197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:30.984512 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 23:44:30.984742 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 23:44:30.985125 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":360,"threads_started":1}
I20260808 23:44:30.987578 5876 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41849
I20260808 23:44:30.987867 5876 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:30.988565 5876 heartbeater.cc:511] Master 127.18.186.190:41849 requested a full tablet report, sending...
I20260808 23:44:30.989961 5389 ts_manager.cc:194] Registered new tserver with Master: 5c707bc7e04047a2ac087f53d03955f5 (127.18.186.131:41739)
I20260808 23:44:30.990808 5389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:45357
I20260808 23:44:30.994210 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:44:31.011794 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:30.998771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58758 (local address 127.18.186.190:41849)
0808 23:44:30.998936 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:30.998940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:30.999409 (+ 469us) server_negotiation.cc:407] Connection header received
0808 23:44:30.999628 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:30.999631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:30.999701 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:30.999781 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:31.001239 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.001930 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.003345 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.003810 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.003944 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.006492 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:31.006497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:31.006500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:31.006532 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:31.009076 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:31.009758 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:31.009766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:31.009770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:31.009895 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:31.010286 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:31.010294 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:31.010298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:31.010628 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:31.010888 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.011219 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.011480 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:44:31.016045 5389 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:44:31.023053 5389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:44:31.051273 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.036233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.18.186.129:46049)
0808 23:44:31.039077 (+ 2844us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.039087 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.039690 (+ 603us) server_negotiation.cc:407] Connection header received
0808 23:44:31.040023 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.040032 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.040315 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.040532 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:31.041719 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.043515 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.047087 (+ 3572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.049195 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.049420 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.049675 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.050598 (+ 923us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.050853 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2656,"thread_start_us":2491,"threads_started":1}
I20260808 23:44:31.052182 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.035475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.18.186.131:41739)
0808 23:44:31.037599 (+ 2124us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.037612 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.038508 (+ 896us) server_negotiation.cc:407] Connection header received
0808 23:44:31.039505 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.039516 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.039794 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.040017 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:31.041396 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.043607 (+ 2211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.048160 (+ 4553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.050189 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.050410 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.050627 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.051446 (+ 819us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.051685 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1869,"thread_start_us":1648,"threads_started":1}
I20260808 23:44:31.052528 5890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.035367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46796)
0808 23:44:31.039544 (+ 4177us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.039560 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:31.039805 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:31.040898 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:31.040906 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:31.040942 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:31.041498 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.041512 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.043749 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.043758 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.046881 (+ 3123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.046895 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.049924 (+ 3029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.049933 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.050248 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.050320 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 23:44:31.051799 (+ 1479us) client_negotiation.cc:240] Negotiation successful
0808 23:44:31.052028 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4021,"thread_start_us":2462,"threads_started":1}
I20260808 23:44:31.053691 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.035153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:41739 (local address 127.0.0.1:45410)
0808 23:44:31.038203 (+ 3050us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.038240 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:31.039140 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:31.040519 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:31.040530 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:31.040572 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:31.041089 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.041110 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.044883 (+ 3773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.044891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.047920 (+ 3029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.047940 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.050867 (+ 2927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.050873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.051141 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.051192 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 23:44:31.052956 (+ 1764us) client_negotiation.cc:240] Negotiation successful
0808 23:44:31.053168 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2815,"thread_start_us":4099,"threads_started":1}
I20260808 23:44:31.057404 5518 tablet_service.cc:1467] Processing CreateTablet for tablet deec7fa94aae4356b6a5a54f70d7d60c (DEFAULT_TABLE table=test-table [id=e15da38199a6480fbca37c536a122878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:31.059013 5518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deec7fa94aae4356b6a5a54f70d7d60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:31.058797 5809 tablet_service.cc:1467] Processing CreateTablet for tablet deec7fa94aae4356b6a5a54f70d7d60c (DEFAULT_TABLE table=test-table [id=e15da38199a6480fbca37c536a122878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:31.060355 5809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deec7fa94aae4356b6a5a54f70d7d60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:31.061702 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.039701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:42011 (local address 127.0.0.1:52886)
0808 23:44:31.054364 (+ 14663us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.054401 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:31.054532 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:31.056089 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:31.056093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:31.056114 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:31.056241 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.056248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.058230 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.058237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.059573 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.059586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.060926 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.060931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.061120 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.061159 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 23:44:31.061270 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 23:44:31.061416 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14472,"thread_start_us":15131,"threads_started":1}
I20260808 23:44:31.061830 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.039998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.18.186.130:42011)
0808 23:44:31.055469 (+ 15471us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.055478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.055507 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:44:31.055603 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.055611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.055860 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.056385 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:31.056645 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.058095 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.059722 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.060734 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.060915 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.061084 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.061314 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.061492 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15264,"thread_start_us":15120,"threads_started":1}
I20260808 23:44:31.065443 5665 tablet_service.cc:1467] Processing CreateTablet for tablet deec7fa94aae4356b6a5a54f70d7d60c (DEFAULT_TABLE table=test-table [id=e15da38199a6480fbca37c536a122878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:31.066569 5665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deec7fa94aae4356b6a5a54f70d7d60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:31.074921 5897 tablet_bootstrap.cc:492] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: Bootstrap starting.
I20260808 23:44:31.077383 5899 tablet_bootstrap.cc:492] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Bootstrap starting.
I20260808 23:44:31.078817 5897 tablet_bootstrap.cc:654] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:31.078969 5900 tablet_bootstrap.cc:492] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Bootstrap starting.
I20260808 23:44:31.080308 5897 log.cc:824] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:31.081355 5899 tablet_bootstrap.cc:654] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:31.082621 5900 tablet_bootstrap.cc:654] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:31.082623 5899 log.cc:824] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:31.082990 5897 tablet_bootstrap.cc:492] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: No bootstrap required, opened a new log
I20260808 23:44:31.083303 5897 ts_tablet_manager.cc:1397] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 23:44:31.083942 5900 log.cc:824] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:31.086009 5899 tablet_bootstrap.cc:492] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: No bootstrap required, opened a new log
I20260808 23:44:31.086295 5899 ts_tablet_manager.cc:1397] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 23:44:31.088357 5900 tablet_bootstrap.cc:492] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: No bootstrap required, opened a new log
I20260808 23:44:31.088676 5900 ts_tablet_manager.cc:1397] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 23:44:31.095363 5897 raft_consensus.cc:348] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.095909 5897 raft_consensus.cc:374] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:31.095743 5900 raft_consensus.cc:348] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.096024 5897 raft_consensus.cc:729] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8971b1698b462ea5bf6ed8d588dc25, State: Initialized, Role: FOLLOWER
I20260808 23:44:31.096177 5900 raft_consensus.cc:374] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:31.096282 5900 raft_consensus.cc:729] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc14ef33f28e4a758b0b609c7b420c5e, State: Initialized, Role: FOLLOWER
I20260808 23:44:31.096823 5897 consensus_queue.cc:260] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.096938 5900 consensus_queue.cc:260] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.098331 5899 raft_consensus.cc:348] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.098953 5899 raft_consensus.cc:374] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:31.099099 5899 raft_consensus.cc:729] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c707bc7e04047a2ac087f53d03955f5, State: Initialized, Role: FOLLOWER
I20260808 23:44:31.100015 5899 consensus_queue.cc:260] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.101552 5876 heartbeater.cc:503] Master 127.18.186.190:41849 was elected leader, sending a full tablet report...
I20260808 23:44:31.102306 5899 ts_tablet_manager.cc:1428] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260808 23:44:31.103875 5733 heartbeater.cc:503] Master 127.18.186.190:41849 was elected leader, sending a full tablet report...
I20260808 23:44:31.105386 5897 ts_tablet_manager.cc:1428] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: Time spent starting tablet: real 0.022s user 0.014s sys 0.008s
I20260808 23:44:31.105819 5900 ts_tablet_manager.cc:1428] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
W20260808 23:44:31.112533 5592 tablet.cc:2369] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:44:31.156657 5734 tablet.cc:2369] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:31.187857 5905 raft_consensus.cc:482] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:44:31.188200 5905 raft_consensus.cc:504] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.190178 5905 leader_election.cc:288] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c707bc7e04047a2ac087f53d03955f5 (127.18.186.131:41739), cc14ef33f28e4a758b0b609c7b420c5e (127.18.186.130:42011)
I20260808 23:44:31.197095 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.190707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:42533 (local address 127.18.186.131:41739)
0808 23:44:31.190982 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.190992 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.191480 (+ 488us) server_negotiation.cc:407] Connection header received
0808 23:44:31.191775 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.191785 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.191884 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.191997 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:31.192577 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.193332 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.195097 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.196049 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.196223 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.196304 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.196679 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.196809 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 23:44:31.197165 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.190625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:41739 (local address 127.18.186.129:42533)
0808 23:44:31.191356 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.191373 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:31.191585 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:31.192054 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:31.192060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:31.192087 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:31.192420 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.192429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.193474 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.193480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.194933 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.194945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.196263 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.196269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.196475 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.196522 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 23:44:31.196651 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 23:44:31.196805 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":427,"threads_started":1}
I20260808 23:44:31.197373 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.191723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:41697 (local address 127.18.186.130:42011)
0808 23:44:31.191967 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.191974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.192253 (+ 279us) server_negotiation.cc:407] Connection header received
0808 23:44:31.192473 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.192478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.192537 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.192629 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:31.193175 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.193867 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.195634 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.196381 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.196567 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.196641 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.197018 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.197145 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 23:44:31.197736 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.191544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:42011 (local address 127.18.186.129:41697)
0808 23:44:31.192148 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.192161 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:31.192328 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:31.192657 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:31.192661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:31.192686 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:31.193023 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.193031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.194009 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.194014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.195475 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:31.195492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.196603 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.196610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:31.196801 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.196837 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:44:31.196989 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 23:44:31.197111 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":1,"thread_start_us":290,"threads_started":1}
I20260808 23:44:31.198235 5831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deec7fa94aae4356b6a5a54f70d7d60c" candidate_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c707bc7e04047a2ac087f53d03955f5" is_pre_election: true
I20260808 23:44:31.198753 5831 raft_consensus.cc:2454] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e8971b1698b462ea5bf6ed8d588dc25 in term 0.
I20260808 23:44:31.198678 5686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deec7fa94aae4356b6a5a54f70d7d60c" candidate_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" is_pre_election: true
I20260808 23:44:31.199158 5686 raft_consensus.cc:2454] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e8971b1698b462ea5bf6ed8d588dc25 in term 0.
I20260808 23:44:31.199504 5469 leader_election.cc:302] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c707bc7e04047a2ac087f53d03955f5, 9e8971b1698b462ea5bf6ed8d588dc25; no voters:
I20260808 23:44:31.199918 5905 raft_consensus.cc:2781] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:44:31.200069 5905 raft_consensus.cc:482] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:44:31.200183 5905 raft_consensus.cc:3037] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:31.202378 5905 raft_consensus.cc:504] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.202946 5905 leader_election.cc:288] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [CANDIDATE]: Term 1 election: Requested vote from peers 5c707bc7e04047a2ac087f53d03955f5 (127.18.186.131:41739), cc14ef33f28e4a758b0b609c7b420c5e (127.18.186.130:42011)
I20260808 23:44:31.203397 5831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deec7fa94aae4356b6a5a54f70d7d60c" candidate_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c707bc7e04047a2ac087f53d03955f5"
I20260808 23:44:31.203466 5686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deec7fa94aae4356b6a5a54f70d7d60c" candidate_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc14ef33f28e4a758b0b609c7b420c5e"
I20260808 23:44:31.203645 5831 raft_consensus.cc:3037] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:31.203657 5686 raft_consensus.cc:3037] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:31.205978 5686 raft_consensus.cc:2454] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e8971b1698b462ea5bf6ed8d588dc25 in term 1.
I20260808 23:44:31.206218 5831 raft_consensus.cc:2454] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e8971b1698b462ea5bf6ed8d588dc25 in term 1.
I20260808 23:44:31.206391 5469 leader_election.cc:302] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e8971b1698b462ea5bf6ed8d588dc25, cc14ef33f28e4a758b0b609c7b420c5e; no voters:
I20260808 23:44:31.206668 5905 raft_consensus.cc:2781] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:31.207113 5905 raft_consensus.cc:686] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 1 LEADER]: Becoming Leader. State: Replica: 9e8971b1698b462ea5bf6ed8d588dc25, State: Running, Role: LEADER
I20260808 23:44:31.207650 5905 consensus_queue.cc:237] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } }
I20260808 23:44:31.212399 5388 catalog_manager.cc:5495] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e8971b1698b462ea5bf6ed8d588dc25 (127.18.186.129). New cstate: current_term: 1 leader_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e8971b1698b462ea5bf6ed8d588dc25" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 46049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 } health_report { overall_health: UNKNOWN } } }
W20260808 23:44:31.215379 5877 tablet.cc:2369] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:31.238945 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.232715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46800 (local address 127.18.186.129:46049)
0808 23:44:31.232926 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.232931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.232949 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:44:31.233030 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.233034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.233124 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.233262 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:44:31.233725 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.234492 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.236586 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.237016 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.237117 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.237715 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:44:31.238296 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:44:31.238412 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.238574 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.238644 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:44:31.248097 5686 raft_consensus.cc:1261] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 1 FOLLOWER]: Refusing update from remote peer 9e8971b1698b462ea5bf6ed8d588dc25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:31.248178 5831 raft_consensus.cc:1261] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 1 FOLLOWER]: Refusing update from remote peer 9e8971b1698b462ea5bf6ed8d588dc25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:31.249054 5910 consensus_queue.cc:1035] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc14ef33f28e4a758b0b609c7b420c5e" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 42011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:31.249447 5905 consensus_queue.cc:1035] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c707bc7e04047a2ac087f53d03955f5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 41739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:31.261050 5913 mvcc.cc:205] Tried to move back new op lower bound from 7316409021422116864 to 7316409021270061056. Current Snapshot: MvccSnapshot[applied={T|T < 7316409021422116864}]
I20260808 23:44:31.275413 5389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:31.275779 5389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:31.280074 5389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa2da32defaf4672821dc5a1c638ed20: Sending DeleteTablet for 3 replicas of tablet deec7fa94aae4356b6a5a54f70d7d60c
I20260808 23:44:31.281288 5518 tablet_service.cc:1514] Processing DeleteTablet for tablet deec7fa94aae4356b6a5a54f70d7d60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:31 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:46796
I20260808 23:44:31.281442 5665 tablet_service.cc:1514] Processing DeleteTablet for tablet deec7fa94aae4356b6a5a54f70d7d60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:31 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:52886
I20260808 23:44:31.281702 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet deec7fa94aae4356b6a5a54f70d7d60c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:31 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:45410
I20260808 23:44:31.283532 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 5451
I20260808 23:44:31.293442 5924 tablet_replica.cc:330] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: stopping tablet replica
I20260808 23:44:31.294211 5924 raft_consensus.cc:2229] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:31.294987 5924 raft_consensus.cc:2258] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:31.296012 5925 tablet_replica.cc:330] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: stopping tablet replica
I20260808 23:44:31.296383 5925 raft_consensus.cc:2229] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:31.296804 5925 raft_consensus.cc:2258] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:31.298784 5924 ts_tablet_manager.cc:1905] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:31.304551 5923 tablet_replica.cc:330] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: stopping tablet replica
I20260808 23:44:31.306150 5925 ts_tablet_manager.cc:1905] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 23:44:31 dist-test-slave-5k54 krb5kdc[5338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:44:31.308683 5923 raft_consensus.cc:2229] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:44:31.309526 5923 raft_consensus.cc:2258] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:31.311982 5923 ts_tablet_manager.cc:1905] T deec7fa94aae4356b6a5a54f70d7d60c P 9e8971b1698b462ea5bf6ed8d588dc25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:31.316557 5924 ts_tablet_manager.cc:1918] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:31.316690 5925 ts_tablet_manager.cc:1918] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:31.316794 5924 log.cc:1196] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/deec7fa94aae4356b6a5a54f70d7d60c
I20260808 23:44:31.316957 5925 log.cc:1196] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/deec7fa94aae4356b6a5a54f70d7d60c
I20260808 23:44:31.317500 5924 ts_tablet_manager.cc:1939] T deec7fa94aae4356b6a5a54f70d7d60c P cc14ef33f28e4a758b0b609c7b420c5e: Deleting consensus metadata
I20260808 23:44:31.317641 5925 ts_tablet_manager.cc:1939] T deec7fa94aae4356b6a5a54f70d7d60c P 5c707bc7e04047a2ac087f53d03955f5: Deleting consensus metadata
I20260808 23:44:31.318538 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.285024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46802 (local address 127.18.186.129:46049)
0808 23:44:31.285194 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.285201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.291148 (+ 5947us) server_negotiation.cc:407] Connection header received
0808 23:44:31.291460 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.291466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.291577 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.291694 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:31.298427 (+ 6733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.299707 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.301789 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.302588 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.302708 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.309534 (+ 6826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:31.309549 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:31.309566 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:31.309622 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:31.313879 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:31.314970 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:31.314979 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:31.314985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:31.315188 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:31.315723 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:31.315728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:31.315732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:31.316205 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:31.316552 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.317911 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.318175 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:44:31.319469 5585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:31.320233 5373 catalog_manager.cc:4841] TS cc14ef33f28e4a758b0b609c7b420c5e (127.18.186.130:42011): tablet deec7fa94aae4356b6a5a54f70d7d60c (table test-table [id=e15da38199a6480fbca37c536a122878]) successfully deleted
I20260808 23:44:31.320633 5373 catalog_manager.cc:4841] TS 5c707bc7e04047a2ac087f53d03955f5 (127.18.186.131:41739): tablet deec7fa94aae4356b6a5a54f70d7d60c (table test-table [id=e15da38199a6480fbca37c536a122878]) successfully deleted
I20260808 23:44:31.423530 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 5451
W20260808 23:44:31.445341 5376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.186.129:46049 (error 108)
I20260808 23:44:31.446179 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 5597
I20260808 23:44:31.447523 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.446975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46810)
0808 23:44:31.447213 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.447317 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260808 23:44:31.447743 5376 catalog_manager.cc:4584] TS 9e8971b1698b462ea5bf6ed8d588dc25 (127.18.186.129:46049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet deec7fa94aae4356b6a5a54f70d7d60c: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Aug 08 23:44:31 dist-test-slave-5k54 krb5kdc[5338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:44:31.462574 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.447104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.18.186.130:42011)
0808 23:44:31.447280 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.447287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.447363 (+ 76us) server_negotiation.cc:407] Connection header received
0808 23:44:31.447593 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.447599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.447683 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.447782 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:31.449090 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.449850 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.451224 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.451931 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.452071 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.456334 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:31.456348 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:31.456362 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:31.456403 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:31.459597 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:31.460324 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:31.460334 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:31.460340 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:31.460475 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:31.460874 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:31.460881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:31.460884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:31.461305 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:31.461636 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.461959 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.462193 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:44:31.463316 5727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:31.501541 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.500993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46818)
0808 23:44:31.501229 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.501363 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 23:44:31.502717 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.502275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46828)
0808 23:44:31.502446 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.502532 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 23:44:31.559927 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 5597
I20260808 23:44:31.563540 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.563036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46842)
0808 23:44:31.563246 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.563344 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 23:44:31.564615 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.564258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46858)
0808 23:44:31.564423 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.564502 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:44:31.580176 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 5739
Aug 08 23:44:31 dist-test-slave-5k54 krb5kdc[5338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:44:31.596067 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.580951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.18.186.131:41739)
0808 23:44:31.581145 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:31.581151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:31.581172 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:44:31.581388 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:31.581393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:31.581472 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:31.581579 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:31.582856 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.583671 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.584971 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:31.585389 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:31.585524 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:31.590023 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:31.590039 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:31.590055 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:31.590105 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:31.593305 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:31.593952 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:31.593961 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:31.593964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:31.594119 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:31.594495 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:31.594501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:31.594504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:31.594866 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:31.595165 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:31.595516 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 23:44:31.595742 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:44:31.596701 5871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:31.645336 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.644724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46874)
0808 23:44:31.645021 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.645140 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260808 23:44:31.646525 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.646029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46888)
0808 23:44:31.646247 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.646359 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 23:44:31.686628 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 5739
I20260808 23:44:31.706043 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 5353
I20260808 23:44:31.707293 5419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:31.820379 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:31.819750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:46049 (local address 127.0.0.1:46902)
0808 23:44:31.820035 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:31.820172 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 23:44:31.820395 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 5353
2026-08-08T23:44:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4529 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T23:44:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:31Z Disabled control of system clock
I20260808 23:44:31.911293 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44851
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:44851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:44:32.245743 5936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:32.246042 5936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:32.246213 5936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:32.253298 5936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:32.253443 5936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:32.253533 5936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:32.253641 5936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:32.253718 5936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:32.262678 5936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:44851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:32.263955 5936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:32.265830 5936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:32.277894 5942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:32.277932 5941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:32.277973 5944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:32.278771 5936 server_base.cc:1034] running on GCE node
I20260808 23:44:32.279668 5936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:32.281175 5936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:32.282414 5936 hybrid_clock.cc:648] HybridClock initialized: now 1786232672282358 us; error 51 us; skew 500 ppm
I20260808 23:44:32.282769 5936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:32.288395 5936 webserver.cc:460] Webserver started at http://127.18.186.190:42519/ using document root <none> and password file <none>
I20260808 23:44:32.289189 5936 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:32.289347 5936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:32.289692 5936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:32.292996 5936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "cf91c3bb6c1c4d43b17a27c5637e760f"
format_stamp: "Formatted at 2026-08-08 23:44:32 on dist-test-slave-5k54"
server_key: "4f9e19305c2bbaaa1735dcac92210f0c"
server_key_iv: "6c0325fa37d7f4613a188c7a8444ff88"
server_key_version: "encryptionkey@0"
I20260808 23:44:32.293924 5936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "cf91c3bb6c1c4d43b17a27c5637e760f"
format_stamp: "Formatted at 2026-08-08 23:44:32 on dist-test-slave-5k54"
server_key: "4f9e19305c2bbaaa1735dcac92210f0c"
server_key_iv: "6c0325fa37d7f4613a188c7a8444ff88"
server_key_version: "encryptionkey@0"
I20260808 23:44:32.299543 5936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 23:44:32.303133 5950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:32.304555 5936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:32.304754 5936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "cf91c3bb6c1c4d43b17a27c5637e760f"
format_stamp: "Formatted at 2026-08-08 23:44:32 on dist-test-slave-5k54"
server_key: "4f9e19305c2bbaaa1735dcac92210f0c"
server_key_iv: "6c0325fa37d7f4613a188c7a8444ff88"
server_key_version: "encryptionkey@0"
I20260808 23:44:32.304987 5936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:32.332296 5936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:32.333110 5936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:32.333397 5936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:32.351979 5936 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:44851
I20260808 23:44:32.351997 6003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:44851 every 8 connection(s)
I20260808 23:44:32.353295 5936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:32.356541 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 5936
I20260808 23:44:32.356732 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:32.357295 6004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:32.357372 19178 external_mini_cluster.cc:1428] Setting key 65b4331a760190803d1ff686b80b2526
I20260808 23:44:32.371034 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:32.359245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.18.186.190:44851)
0808 23:44:32.360300 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:32.360320 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:32.360364 (+ 44us) server_negotiation.cc:407] Connection header received
0808 23:44:32.362034 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:32.362055 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:32.362485 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:32.362793 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:32.363413 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:32.364787 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:32.366665 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:32.367314 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:32.367522 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:32.368883 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:32.368890 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:32.368902 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:32.369432 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:32.369505 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:32.369531 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:32.369614 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:32.369841 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 23:44:32.370158 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":426,"threads_started":1}
I20260808 23:44:32.371078 6004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f: Bootstrap starting.
I20260808 23:44:32.374145 6004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:32.375115 6004 log.cc:824] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:32.377765 6004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f: No bootstrap required, opened a new log
I20260808 23:44:32.384621 6004 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf91c3bb6c1c4d43b17a27c5637e760f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44851 } }
I20260808 23:44:32.384992 6004 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:32.385104 6004 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf91c3bb6c1c4d43b17a27c5637e760f, State: Initialized, Role: FOLLOWER
I20260808 23:44:32.385702 6004 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf91c3bb6c1c4d43b17a27c5637e760f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44851 } }
I20260808 23:44:32.385933 6004 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:32.386063 6004 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:32.386230 6004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:32.388374 6004 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf91c3bb6c1c4d43b17a27c5637e760f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44851 } }
I20260808 23:44:32.388885 6004 leader_election.cc:302] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf91c3bb6c1c4d43b17a27c5637e760f; no voters:
I20260808 23:44:32.389441 6004 leader_election.cc:288] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:32.389592 6009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:32.390340 6009 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [term 1 LEADER]: Becoming Leader. State: Replica: cf91c3bb6c1c4d43b17a27c5637e760f, State: Running, Role: LEADER
I20260808 23:44:32.390879 6009 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf91c3bb6c1c4d43b17a27c5637e760f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44851 } }
I20260808 23:44:32.391608 6004 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:32.393671 6011 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf91c3bb6c1c4d43b17a27c5637e760f. Latest consensus state: current_term: 1 leader_uuid: "cf91c3bb6c1c4d43b17a27c5637e760f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf91c3bb6c1c4d43b17a27c5637e760f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44851 } } }
I20260808 23:44:32.394133 6011 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:32.393703 6010 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf91c3bb6c1c4d43b17a27c5637e760f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf91c3bb6c1c4d43b17a27c5637e760f" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44851 } } }
I20260808 23:44:32.394825 6010 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:32.396730 6016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:32.403256 6016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:32.411917 6016 catalog_manager.cc:1347] Generated new cluster ID: acd728c2637146df9fb386337bb589e7
I20260808 23:44:32.412039 6016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:32.425473 6016 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:32.427193 6016 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:32.436168 6016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f: Generated new TSK 0
I20260808 23:44:32.437165 6016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:44:32.457505 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44851
--builtin_ntp_servers=127.18.186.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:44:32.803234 6029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:32.803570 6029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:32.803733 6029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:32.811254 6029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:32.811352 6029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:32.811532 6029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:44:32.820887 6029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:32.822188 6029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:32.824045 6029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:32.837306 6035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:32.837282 6038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:32.837436 6036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:32.837947 6029 server_base.cc:1034] running on GCE node
I20260808 23:44:32.838770 6029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:32.839807 6029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:32.841017 6029 hybrid_clock.cc:648] HybridClock initialized: now 1786232672840975 us; error 33 us; skew 500 ppm
I20260808 23:44:32.841331 6029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:32.843788 6029 webserver.cc:460] Webserver started at http://127.18.186.129:34445/ using document root <none> and password file <none>
I20260808 23:44:32.844493 6029 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:32.844606 6029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:32.844946 6029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:32.847905 6029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "df63928fc4d04af6aec991188d5b79ae"
format_stamp: "Formatted at 2026-08-08 23:44:32 on dist-test-slave-5k54"
server_key: "c586c88bff230e4ae6b782328aa4218e"
server_key_iv: "ec04942451f0eb33ab1f2723bd7ad365"
server_key_version: "encryptionkey@0"
I20260808 23:44:32.848707 6029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "df63928fc4d04af6aec991188d5b79ae"
format_stamp: "Formatted at 2026-08-08 23:44:32 on dist-test-slave-5k54"
server_key: "c586c88bff230e4ae6b782328aa4218e"
server_key_iv: "ec04942451f0eb33ab1f2723bd7ad365"
server_key_version: "encryptionkey@0"
I20260808 23:44:32.853801 6029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:44:32.857189 6045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:32.858469 6029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:44:32.858644 6029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "df63928fc4d04af6aec991188d5b79ae"
format_stamp: "Formatted at 2026-08-08 23:44:32 on dist-test-slave-5k54"
server_key: "c586c88bff230e4ae6b782328aa4218e"
server_key_iv: "ec04942451f0eb33ab1f2723bd7ad365"
server_key_version: "encryptionkey@0"
I20260808 23:44:32.858851 6029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:32.877692 6029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:32.878749 6029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:32.879097 6029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:32.880347 6029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:32.881810 6029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:32.881939 6029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:32.882072 6029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:32.882179 6029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:32.916977 6029 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:36493
I20260808 23:44:32.917037 6164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:36493 every 8 connection(s)
I20260808 23:44:32.918288 6029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:44:32.927649 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 6029
I20260808 23:44:32.927860 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:44:32.928349 19178 external_mini_cluster.cc:1428] Setting key eface2a1d5092460cc9da818a08e0ba4
I20260808 23:44:32.931041 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:32.920830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:45835 (local address 127.18.186.190:44851)
0808 23:44:32.921292 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:32.921297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:32.922421 (+ 1124us) server_negotiation.cc:407] Connection header received
0808 23:44:32.924291 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:32.924296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:32.924390 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:32.924513 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:32.925583 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:32.926353 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:32.928119 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:32.928572 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:32.928706 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:32.930250 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:32.930256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:32.930260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:32.930282 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:32.930331 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:32.930348 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:32.930448 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:32.930701 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 23:44:32.930772 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":250,"threads_started":1}
I20260808 23:44:32.931490 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44851
--builtin_ntp_servers=127.18.186.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:44:32.932456 6168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:32.921340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44851 (local address 127.18.186.129:45835)
0808 23:44:32.922241 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:32.922303 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:32.924020 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:32.924751 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:32.924772 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:32.924858 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:32.925414 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:32.925431 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:32.926518 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:32.926523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:32.927964 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:32.927976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:32.928795 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:32.928805 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:32.929082 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:32.929884 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:32.929886 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:32.929923 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:32.930038 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:32.930475 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:32.930488 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:32.930496 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:44:32.930688 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 23:44:32.931097 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":270,"threads_started":1}
I20260808 23:44:32.933957 6165 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44851
I20260808 23:44:32.934255 6165 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:32.934986 6165 heartbeater.cc:511] Master 127.18.186.190:44851 requested a full tablet report, sending...
I20260808 23:44:32.937083 5968 ts_manager.cc:194] Registered new tserver with Master: df63928fc4d04af6aec991188d5b79ae (127.18.186.129:36493)
I20260808 23:44:32.938476 5968 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.129:45835
W20260808 23:44:33.265177 6169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:33.265445 6169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:33.265530 6169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:33.272495 6169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:33.272586 6169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:33.272687 6169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:44:33.281308 6169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:33.282510 6169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:33.284193 6169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:33.296940 6176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:33.296957 6178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:33.297011 6175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:33.297588 6169 server_base.cc:1034] running on GCE node
I20260808 23:44:33.298300 6169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:33.299269 6169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:33.300496 6169 hybrid_clock.cc:648] HybridClock initialized: now 1786232673300405 us; error 77 us; skew 500 ppm
I20260808 23:44:33.300856 6169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:33.303190 6169 webserver.cc:460] Webserver started at http://127.18.186.130:38157/ using document root <none> and password file <none>
I20260808 23:44:33.303925 6169 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:33.304044 6169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:33.304373 6169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:33.307181 6169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "50bc7491480a4da5a4524dce8ff6c8d5"
format_stamp: "Formatted at 2026-08-08 23:44:33 on dist-test-slave-5k54"
server_key: "9d72471fa71453358c1e84f374e9e75b"
server_key_iv: "b2f0d312c2c522f392f6c84c67792e02"
server_key_version: "encryptionkey@0"
I20260808 23:44:33.308012 6169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "50bc7491480a4da5a4524dce8ff6c8d5"
format_stamp: "Formatted at 2026-08-08 23:44:33 on dist-test-slave-5k54"
server_key: "9d72471fa71453358c1e84f374e9e75b"
server_key_iv: "b2f0d312c2c522f392f6c84c67792e02"
server_key_version: "encryptionkey@0"
I20260808 23:44:33.313256 6169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:44:33.316707 6185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:33.318186 6169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:44:33.318327 6169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "50bc7491480a4da5a4524dce8ff6c8d5"
format_stamp: "Formatted at 2026-08-08 23:44:33 on dist-test-slave-5k54"
server_key: "9d72471fa71453358c1e84f374e9e75b"
server_key_iv: "b2f0d312c2c522f392f6c84c67792e02"
server_key_version: "encryptionkey@0"
I20260808 23:44:33.318506 6169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:33.331967 6169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:33.332650 6169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:33.332896 6169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:33.333729 6169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:33.335093 6169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:33.335179 6169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:33.335259 6169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:33.335322 6169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:33.370428 6169 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:44551
I20260808 23:44:33.370473 6304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:44551 every 8 connection(s)
I20260808 23:44:33.371824 6169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:44:33.381191 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 6169
I20260808 23:44:33.381397 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:44:33.381873 19178 external_mini_cluster.cc:1428] Setting key b7586d358d3e791fa634aed95ec3cd71
I20260808 23:44:33.384141 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.374324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:42539 (local address 127.18.186.190:44851)
0808 23:44:33.374500 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:33.374504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:33.375853 (+ 1349us) server_negotiation.cc:407] Connection header received
0808 23:44:33.377629 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:33.377634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:33.377719 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:33.377842 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:33.378902 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.379687 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.381210 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.381742 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.381874 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.383308 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:33.383313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:33.383316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:33.383335 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:33.383366 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:33.383379 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:33.383520 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:33.383741 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 23:44:33.383840 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:44:33.384831 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44851
--builtin_ntp_servers=127.18.186.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:44:33.385224 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.374746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44851 (local address 127.18.186.130:42539)
0808 23:44:33.375687 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:33.375727 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:33.377348 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:33.378076 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:33.378085 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:33.378170 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:33.378726 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.378742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.379847 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.379853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.381056 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.381068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.381972 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.381979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.382203 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.382977 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:33.382980 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:33.383018 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:33.383089 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:33.383520 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:33.383535 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:33.383545 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 23:44:33.383730 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 23:44:33.384109 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":273,"threads_started":1}
I20260808 23:44:33.386600 6305 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44851
I20260808 23:44:33.386904 6305 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:33.387676 6305 heartbeater.cc:511] Master 127.18.186.190:44851 requested a full tablet report, sending...
I20260808 23:44:33.389041 5968 ts_manager.cc:194] Registered new tserver with Master: 50bc7491480a4da5a4524dce8ff6c8d5 (127.18.186.130:44551)
I20260808 23:44:33.389835 5968 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.130:42539
W20260808 23:44:33.724838 6309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:33.725099 6309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:33.725185 6309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:33.732144 6309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:33.732239 6309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:33.732350 6309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:44:33.741122 6309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:44851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:33.742287 6309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:33.744155 6309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:33.757156 6315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:33.757103 6318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:33.757644 6314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:33.758138 6309 server_base.cc:1034] running on GCE node
I20260808 23:44:33.758733 6309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:33.759654 6309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:33.760871 6309 hybrid_clock.cc:648] HybridClock initialized: now 1786232673760820 us; error 62 us; skew 500 ppm
I20260808 23:44:33.761184 6309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:33.763420 6309 webserver.cc:460] Webserver started at http://127.18.186.131:46557/ using document root <none> and password file <none>
I20260808 23:44:33.764156 6309 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:33.764242 6309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:33.764499 6309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:33.767283 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "e1564daac5ef433ca4a5245e66f2f26a"
format_stamp: "Formatted at 2026-08-08 23:44:33 on dist-test-slave-5k54"
server_key: "d5b8af07aa6d74e5d128dac3b2c90413"
server_key_iv: "f542d92d4cef5e6496b585e0713045bc"
server_key_version: "encryptionkey@0"
I20260808 23:44:33.768149 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "e1564daac5ef433ca4a5245e66f2f26a"
format_stamp: "Formatted at 2026-08-08 23:44:33 on dist-test-slave-5k54"
server_key: "d5b8af07aa6d74e5d128dac3b2c90413"
server_key_iv: "f542d92d4cef5e6496b585e0713045bc"
server_key_version: "encryptionkey@0"
I20260808 23:44:33.772950 6309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:44:33.776192 6324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:33.777426 6309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:33.777554 6309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "e1564daac5ef433ca4a5245e66f2f26a"
format_stamp: "Formatted at 2026-08-08 23:44:33 on dist-test-slave-5k54"
server_key: "d5b8af07aa6d74e5d128dac3b2c90413"
server_key_iv: "f542d92d4cef5e6496b585e0713045bc"
server_key_version: "encryptionkey@0"
I20260808 23:44:33.777709 6309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:33.794157 6309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:33.794862 6309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:33.795125 6309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:33.796119 6309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:33.797435 6309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:33.797525 6309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:33.797607 6309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:33.797663 6309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:33.831236 6309 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:37547
I20260808 23:44:33.831291 6448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:37547 every 8 connection(s)
I20260808 23:44:33.832610 6309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:44:33.834791 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 6309
I20260808 23:44:33.835002 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:44:33.835616 19178 external_mini_cluster.cc:1428] Setting key ff92852d80475ecffb02f0e998e32e39
I20260808 23:44:33.845764 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.835383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:57623 (local address 127.18.186.190:44851)
0808 23:44:33.835619 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:33.835626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:33.837385 (+ 1759us) server_negotiation.cc:407] Connection header received
0808 23:44:33.839572 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:33.839577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:33.839669 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:33.839774 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:33.840808 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.841526 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.843027 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.843506 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.843641 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.844952 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:33.844959 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:33.844967 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:33.844986 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:33.845018 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:33.845030 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:33.845125 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:33.845341 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 23:44:33.845441 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 23:44:33.846792 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.836033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:44851 (local address 127.18.186.131:57623)
0808 23:44:33.837206 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:33.837288 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:33.839280 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:33.840012 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:33.840020 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:33.840104 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:33.840651 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.840666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.841687 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.841692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.842881 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.842892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.843727 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.843734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.843969 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.844624 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:33.844626 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:33.844666 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:33.844745 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:33.845124 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:33.845136 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:33.845146 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 23:44:33.845328 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 23:44:33.845730 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":342,"threads_started":1}
I20260808 23:44:33.848167 6449 heartbeater.cc:347] Connected to a master server at 127.18.186.190:44851
I20260808 23:44:33.848453 6449 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:33.849298 6449 heartbeater.cc:511] Master 127.18.186.190:44851 requested a full tablet report, sending...
I20260808 23:44:33.850960 5967 ts_manager.cc:194] Registered new tserver with Master: e1564daac5ef433ca4a5245e66f2f26a (127.18.186.131:37547)
I20260808 23:44:33.852028 5967 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.131:57623
I20260808 23:44:33.863294 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:44:33.873107 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.867501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53826 (local address 127.18.186.190:44851)
0808 23:44:33.867687 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:33.867691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:33.868168 (+ 477us) server_negotiation.cc:407] Connection header received
0808 23:44:33.868297 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:33.868300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:33.868372 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:33.868450 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:33.868898 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.869661 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.870955 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.871341 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.871494 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.872562 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:33.872566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:33.872569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:33.872587 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:33.872614 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:33.872625 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:33.872693 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:33.872806 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 23:44:33.872884 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:44:33.877027 5967 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:44:33.884085 5967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:44:33.910148 6463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.895920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:51702)
0808 23:44:33.896855 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:33.896908 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:33.897093 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:33.898800 (+ 1707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:33.898812 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:33.898855 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:33.899456 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.899474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.902456 (+ 2982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.902465 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.905765 (+ 3300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.905788 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.909130 (+ 3342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.909136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.909362 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.909428 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 23:44:33.909535 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 23:44:33.909764 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":12330,"threads_started":1}
I20260808 23:44:33.911646 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.896555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51702 (local address 127.18.186.129:36493)
0808 23:44:33.898071 (+ 1516us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:33.898079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:33.898110 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:44:33.898206 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:33.898213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:33.898489 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:33.899721 (+ 1232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:33.900059 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.902260 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.906113 (+ 3853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.908178 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.910660 (+ 2482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.910932 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:33.911006 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 23:44:33.911221 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":1193,"threads_started":1}
I20260808 23:44:33.915777 6095 tablet_service.cc:1467] Processing CreateTablet for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc (DEFAULT_TABLE table=test-table [id=68f452f33baa44eaba87c5cff528915f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:33.917393 6095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:33.917454 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.897751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37547 (local address 127.0.0.1:37876)
0808 23:44:33.908729 (+ 10978us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:33.908750 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:33.908910 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:33.910893 (+ 1983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:33.910897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:33.910923 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:33.911275 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.911287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.913164 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.913172 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.914798 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.914809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.916690 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.916696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.916854 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.916883 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 23:44:33.916972 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:44:33.917108 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10858,"mutex_wait_us":3,"thread_start_us":10714,"threads_started":1}
I20260808 23:44:33.918148 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.896727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47290 (local address 127.18.186.130:44551)
0808 23:44:33.897633 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:33.897644 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:33.908229 (+ 10585us) server_negotiation.cc:407] Connection header received
0808 23:44:33.908849 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:33.908862 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:33.909195 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:33.909406 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:33.910379 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.911504 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.914135 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.915554 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.915710 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.915917 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:33.917332 (+ 1415us) server_negotiation.cc:299] Negotiation successful
0808 23:44:33.917786 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":1359,"threads_started":1}
I20260808 23:44:33.918275 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.898385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37876 (local address 127.18.186.131:37547)
0808 23:44:33.910023 (+ 11638us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:33.910031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:33.910064 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:44:33.910157 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:33.910166 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:33.910422 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:33.910620 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:33.911512 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.913051 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.914969 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.916162 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.916347 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.916566 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:33.917740 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0808 23:44:33.917903 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11457,"thread_start_us":11289,"threads_started":1}
I20260808 23:44:33.919132 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:33.896212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:44551 (local address 127.0.0.1:47290)
0808 23:44:33.908079 (+ 11867us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:33.908104 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:33.908582 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:33.909767 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:33.909771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:33.909795 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:33.910160 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.910169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.911644 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.911649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.913929 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:33.913943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:33.915768 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:33.915775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:33.915940 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:33.915985 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 23:44:33.918678 (+ 2693us) client_negotiation.cc:240] Negotiation successful
0808 23:44:33.918816 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11680,"mutex_wait_us":65,"thread_start_us":679,"threads_started":1}
I20260808 23:44:33.921669 6378 tablet_service.cc:1467] Processing CreateTablet for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc (DEFAULT_TABLE table=test-table [id=68f452f33baa44eaba87c5cff528915f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:33.923178 6378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:33.925166 6236 tablet_service.cc:1467] Processing CreateTablet for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc (DEFAULT_TABLE table=test-table [id=68f452f33baa44eaba87c5cff528915f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:33.926471 6236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:33.937556 6469 tablet_bootstrap.cc:492] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: Bootstrap starting.
I20260808 23:44:33.939671 6471 tablet_bootstrap.cc:492] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Bootstrap starting.
I20260808 23:44:33.941308 6165 heartbeater.cc:503] Master 127.18.186.190:44851 was elected leader, sending a full tablet report...
I20260808 23:44:33.941303 6472 tablet_bootstrap.cc:492] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Bootstrap starting.
I20260808 23:44:33.941514 6469 tablet_bootstrap.cc:654] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:33.942873 6469 log.cc:824] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:33.943207 6471 tablet_bootstrap.cc:654] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:33.944593 6471 log.cc:824] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:33.944648 6472 tablet_bootstrap.cc:654] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:33.946743 6472 log.cc:824] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:33.949615 6472 tablet_bootstrap.cc:492] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: No bootstrap required, opened a new log
I20260808 23:44:33.949894 6472 ts_tablet_manager.cc:1397] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 23:44:33.949988 6469 tablet_bootstrap.cc:492] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: No bootstrap required, opened a new log
I20260808 23:44:33.950150 6471 tablet_bootstrap.cc:492] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: No bootstrap required, opened a new log
I20260808 23:44:33.950258 6469 ts_tablet_manager.cc:1397] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260808 23:44:33.950392 6471 ts_tablet_manager.cc:1397] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260808 23:44:33.957544 6472 raft_consensus.cc:348] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:33.957924 6472 raft_consensus.cc:374] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:33.958032 6472 raft_consensus.cc:729] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50bc7491480a4da5a4524dce8ff6c8d5, State: Initialized, Role: FOLLOWER
I20260808 23:44:33.958675 6472 consensus_queue.cc:260] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:33.959830 6305 heartbeater.cc:503] Master 127.18.186.190:44851 was elected leader, sending a full tablet report...
I20260808 23:44:33.960523 6472 ts_tablet_manager.cc:1428] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260808 23:44:33.960980 6471 raft_consensus.cc:348] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:33.960991 6469 raft_consensus.cc:348] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:33.961403 6469 raft_consensus.cc:374] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:33.961403 6471 raft_consensus.cc:374] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:33.961508 6469 raft_consensus.cc:729] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df63928fc4d04af6aec991188d5b79ae, State: Initialized, Role: FOLLOWER
I20260808 23:44:33.961511 6471 raft_consensus.cc:729] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1564daac5ef433ca4a5245e66f2f26a, State: Initialized, Role: FOLLOWER
I20260808 23:44:33.962116 6469 consensus_queue.cc:260] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:33.962112 6471 consensus_queue.cc:260] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:33.964844 6469 ts_tablet_manager.cc:1428] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260808 23:44:33.965420 6449 heartbeater.cc:503] Master 127.18.186.190:44851 was elected leader, sending a full tablet report...
I20260808 23:44:33.965662 6471 ts_tablet_manager.cc:1428] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260808 23:44:34.084436 6451 tablet.cc:2369] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:44:34.123731 6306 tablet.cc:2369] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:44:34.170754 6166 tablet.cc:2369] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:34.205228 6478 raft_consensus.cc:482] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:44:34.205566 6478 raft_consensus.cc:504] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:34.207235 6478 leader_election.cc:288] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df63928fc4d04af6aec991188d5b79ae (127.18.186.129:36493), 50bc7491480a4da5a4524dce8ff6c8d5 (127.18.186.130:44551)
I20260808 23:44:34.213847 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.207654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.18.186.131:36323)
0808 23:44:34.207965 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.208004 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:34.208159 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:34.208632 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:34.208636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:34.208660 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:34.208970 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:34.208993 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.210250 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.210256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:34.211840 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:34.211852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.213099 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.213105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:34.213274 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:34.213309 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:44:34.213425 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 23:44:34.213567 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":21,"spinlock_wait_cycles":17280}
I20260808 23:44:34.213894 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.208019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:36323 (local address 127.18.186.129:36493)
0808 23:44:34.208238 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:34.208245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:34.208267 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:44:34.208348 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:34.208353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:34.208464 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:34.208570 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:34.209171 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.210108 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.211989 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.212949 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.213065 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:34.213112 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:34.213483 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 23:44:34.213617 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:44:34.214263 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.207744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:57787 (local address 127.18.186.130:44551)
0808 23:44:34.207986 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:34.207994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:34.208729 (+ 735us) server_negotiation.cc:407] Connection header received
0808 23:44:34.208944 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:34.208949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:34.209037 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:34.209131 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:34.209729 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.210552 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.212489 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.213318 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.213497 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:34.213564 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:34.213867 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 23:44:34.213993 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:44:34.214594 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.207739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:44551 (local address 127.18.186.131:57787)
0808 23:44:34.208596 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.208626 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:34.208790 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:34.209184 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:34.209188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:34.209212 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:34.209562 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:34.209573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.210701 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.210705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:34.212293 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:34.212307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.213508 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.213514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:34.213691 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:34.213723 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:44:34.213815 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 23:44:34.214265 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":3,"thread_start_us":473,"threads_started":1}
I20260808 23:44:34.215020 6118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b1c2e8a8eb4c49ba1252af032bf4bc" candidate_uuid: "e1564daac5ef433ca4a5245e66f2f26a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df63928fc4d04af6aec991188d5b79ae" is_pre_election: true
I20260808 23:44:34.215667 6118 raft_consensus.cc:2454] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1564daac5ef433ca4a5245e66f2f26a in term 0.
I20260808 23:44:34.215683 6257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b1c2e8a8eb4c49ba1252af032bf4bc" candidate_uuid: "e1564daac5ef433ca4a5245e66f2f26a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" is_pre_election: true
I20260808 23:44:34.216233 6257 raft_consensus.cc:2454] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1564daac5ef433ca4a5245e66f2f26a in term 0.
I20260808 23:44:34.216331 6327 leader_election.cc:302] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df63928fc4d04af6aec991188d5b79ae, e1564daac5ef433ca4a5245e66f2f26a; no voters:
I20260808 23:44:34.216825 6478 raft_consensus.cc:2781] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:44:34.216984 6478 raft_consensus.cc:482] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:44:34.217103 6478 raft_consensus.cc:3037] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:34.219327 6478 raft_consensus.cc:504] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:34.219950 6478 leader_election.cc:288] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [CANDIDATE]: Term 1 election: Requested vote from peers df63928fc4d04af6aec991188d5b79ae (127.18.186.129:36493), 50bc7491480a4da5a4524dce8ff6c8d5 (127.18.186.130:44551)
I20260808 23:44:34.220400 6118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b1c2e8a8eb4c49ba1252af032bf4bc" candidate_uuid: "e1564daac5ef433ca4a5245e66f2f26a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df63928fc4d04af6aec991188d5b79ae"
I20260808 23:44:34.220400 6257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b1c2e8a8eb4c49ba1252af032bf4bc" candidate_uuid: "e1564daac5ef433ca4a5245e66f2f26a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50bc7491480a4da5a4524dce8ff6c8d5"
I20260808 23:44:34.220610 6257 raft_consensus.cc:3037] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:34.220610 6118 raft_consensus.cc:3037] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:34.223898 6257 raft_consensus.cc:2454] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1564daac5ef433ca4a5245e66f2f26a in term 1.
I20260808 23:44:34.223898 6118 raft_consensus.cc:2454] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1564daac5ef433ca4a5245e66f2f26a in term 1.
I20260808 23:44:34.224320 6326 leader_election.cc:302] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50bc7491480a4da5a4524dce8ff6c8d5, e1564daac5ef433ca4a5245e66f2f26a; no voters:
I20260808 23:44:34.224618 6478 raft_consensus.cc:2781] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:34.225024 6478 raft_consensus.cc:686] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 1 LEADER]: Becoming Leader. State: Replica: e1564daac5ef433ca4a5245e66f2f26a, State: Running, Role: LEADER
I20260808 23:44:34.225495 6478 consensus_queue.cc:237] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } }
I20260808 23:44:34.230183 5968 catalog_manager.cc:5495] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a reported cstate change: term changed from 0 to 1, leader changed from <none> to e1564daac5ef433ca4a5245e66f2f26a (127.18.186.131). New cstate: current_term: 1 leader_uuid: "e1564daac5ef433ca4a5245e66f2f26a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1564daac5ef433ca4a5245e66f2f26a" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260808 23:44:34.282601 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.277046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37888 (local address 127.18.186.131:37547)
0808 23:44:34.277273 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:34.277277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:34.277295 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:44:34.277474 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:34.277477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:34.277563 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:34.277687 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:44:34.278015 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.278739 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.280150 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.280547 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.280640 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:34.281317 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:44:34.282095 (+ 778us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:44:34.282207 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:34.282289 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 23:44:34.282351 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:44:34.290309 6257 raft_consensus.cc:1261] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 1 FOLLOWER]: Refusing update from remote peer e1564daac5ef433ca4a5245e66f2f26a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:34.290374 6118 raft_consensus.cc:1261] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 1 FOLLOWER]: Refusing update from remote peer e1564daac5ef433ca4a5245e66f2f26a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:34.291306 6481 consensus_queue.cc:1035] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [LEADER]: Connected to new peer: Peer: permanent_uuid: "50bc7491480a4da5a4524dce8ff6c8d5" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 44551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:34.291781 6478 consensus_queue.cc:1035] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df63928fc4d04af6aec991188d5b79ae" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 36493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:34.302966 6484 mvcc.cc:205] Tried to move back new op lower bound from 7316409033883758592 to 7316409033631342592. Current Snapshot: MvccSnapshot[applied={T|T < 7316409033883758592}]
I20260808 23:44:34.315285 5968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:34.315629 5968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:34.319258 5968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf91c3bb6c1c4d43b17a27c5637e760f: Sending DeleteTablet for 3 replicas of tablet b9b1c2e8a8eb4c49ba1252af032bf4bc
I20260808 23:44:34.320425 6095 tablet_service.cc:1514] Processing DeleteTablet for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:34 UTC) from {username='slave'} at 127.0.0.1:51702
I20260808 23:44:34.320853 6378 tablet_service.cc:1514] Processing DeleteTablet for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:34 UTC) from {username='slave'} at 127.0.0.1:37876
I20260808 23:44:34.321508 6236 tablet_service.cc:1514] Processing DeleteTablet for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:34 UTC) from {username='slave'} at 127.0.0.1:47290
I20260808 23:44:34.321864 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 6029
I20260808 23:44:34.325350 6497 tablet_replica.cc:330] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: stopping tablet replica
I20260808 23:44:34.325950 6497 raft_consensus.cc:2229] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:34.326653 6497 raft_consensus.cc:2258] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:34.329058 6497 ts_tablet_manager.cc:1905] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:34.333195 6496 tablet_replica.cc:330] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: stopping tablet replica
I20260808 23:44:34.333619 6496 raft_consensus.cc:2229] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:44:34.334297 6496 raft_consensus.cc:2258] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:34.338129 6493 tablet_replica.cc:330] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: stopping tablet replica
I20260808 23:44:34.338380 6497 ts_tablet_manager.cc:1918] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:34.338586 6493 raft_consensus.cc:2229] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:34.338945 6497 log.cc:1196] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/b9b1c2e8a8eb4c49ba1252af032bf4bc
I20260808 23:44:34.339083 6493 raft_consensus.cc:2258] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:34.339691 6497 ts_tablet_manager.cc:1939] T b9b1c2e8a8eb4c49ba1252af032bf4bc P 50bc7491480a4da5a4524dce8ff6c8d5: Deleting consensus metadata
I20260808 23:44:34.341547 5951 catalog_manager.cc:4841] TS 50bc7491480a4da5a4524dce8ff6c8d5 (127.18.186.130:44551): tablet b9b1c2e8a8eb4c49ba1252af032bf4bc (table test-table [id=68f452f33baa44eaba87c5cff528915f]) successfully deleted
I20260808 23:44:34.341806 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.323049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34288 (local address 127.18.186.129:36493)
0808 23:44:34.323249 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:34.323257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:34.333387 (+ 10130us) server_negotiation.cc:407] Connection header received
0808 23:44:34.333676 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:34.333682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:34.333781 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:34.333861 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:34.334536 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.335794 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.337796 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.338528 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.338634 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:34.340117 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:34.340126 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:34.340148 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:34.340898 (+ 750us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:34.340960 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:34.340988 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:34.341104 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:34.341325 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 23:44:34.341459 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:44:34.341961 6493 ts_tablet_manager.cc:1905] T b9b1c2e8a8eb4c49ba1252af032bf4bc P df63928fc4d04af6aec991188d5b79ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:34.343201 6160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:34.344026 6496 ts_tablet_manager.cc:1905] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:34.351178 6496 ts_tablet_manager.cc:1918] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:34.355654 6496 log.cc:1196] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/b9b1c2e8a8eb4c49ba1252af032bf4bc
I20260808 23:44:34.356262 6496 ts_tablet_manager.cc:1939] T b9b1c2e8a8eb4c49ba1252af032bf4bc P e1564daac5ef433ca4a5245e66f2f26a: Deleting consensus metadata
I20260808 23:44:34.358108 5951 catalog_manager.cc:4841] TS e1564daac5ef433ca4a5245e66f2f26a (127.18.186.131:37547): tablet b9b1c2e8a8eb4c49ba1252af032bf4bc (table test-table [id=68f452f33baa44eaba87c5cff528915f]) successfully deleted
I20260808 23:44:34.453531 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 6029
I20260808 23:44:34.473508 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 6169
W20260808 23:44:34.473519 5952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.186.129:36493 (error 108)
I20260808 23:44:34.477020 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.475595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:34302)
0808 23:44:34.476676 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.476807 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":868,"threads_started":1}
W20260808 23:44:34.477279 5952 catalog_manager.cc:4584] TS df63928fc4d04af6aec991188d5b79ae (127.18.186.129:36493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9b1c2e8a8eb4c49ba1252af032bf4bc: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
I20260808 23:44:34.482129 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.474453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.18.186.130:44551)
0808 23:44:34.474689 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:34.474698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:34.474726 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:44:34.474893 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:34.474901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:34.475048 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:34.475162 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:34.475961 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.476993 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.478467 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.479285 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.479606 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:34.480714 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:34.480728 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:34.480742 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:34.481331 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:34.481418 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:34.481453 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:34.481582 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:34.481689 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 23:44:34.481792 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:44:34.482786 6300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:34.530926 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.530464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:34306)
0808 23:44:34.530680 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.530776 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 23:44:34.532171 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.531769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:34318)
0808 23:44:34.531964 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.532034 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:44:34.592760 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.592241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:34330)
0808 23:44:34.592465 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.592581 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 23:44:34.593900 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.593511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:34340)
0808 23:44:34.593671 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.593749 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 23:44:34.609450 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 6169
I20260808 23:44:34.635317 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 6309
I20260808 23:44:34.643348 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.636299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59080 (local address 127.18.186.131:37547)
0808 23:44:34.636497 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:34.636504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:34.636532 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:44:34.636620 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:34.636630 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:34.636741 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:34.636847 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:34.637216 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.638392 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.639838 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:34.640517 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:34.640646 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:34.641654 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:34.641667 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:34.641685 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:34.642542 (+ 857us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:34.642616 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:34.642646 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:34.642764 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:34.642852 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 23:44:34.642958 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:44:34.644145 6444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:34.674281 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.673787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:34342)
0808 23:44:34.674015 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.674129 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 23:44:34.675415 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.675013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:34356)
0808 23:44:34.675206 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.675274 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260808 23:44:34.765225 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 6309
I20260808 23:44:34.783972 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 5936
I20260808 23:44:34.785086 5999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:34.897490 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 5936
I20260808 23:44:34.897527 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:34.896965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:36493 (local address 127.0.0.1:34360)
0808 23:44:34.897181 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:34.897316 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:36493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
2026-08-08T23:44:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3074 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:44:35 dist-test-slave-5k54 krb5kdc[6507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:44:35 dist-test-slave-5k54 krb5kdc[6507](info): set up 2 sockets
Aug 08 23:44:35 dist-test-slave-5k54 krb5kdc[6507](info): commencing operation
krb5kdc: starting...
W20260808 23:44:36.058648 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:44:36 dist-test-slave-5k54 krb5kdc[6507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:44:36.351228 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42101
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:38839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:36.688086 6521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:36.688360 6521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:36.688529 6521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:36.695679 6521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:36.695782 6521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:36.695883 6521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:36.695982 6521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:36.696060 6521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:36.706112 6521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:38839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:36.707507 6521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:36.708923 6521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:36.721101 6529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:36.721204 6526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:36.721144 6527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:36.721891 6521 server_base.cc:1034] running on GCE node
I20260808 23:44:36.722646 6521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:36.724098 6521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:36.725332 6521 hybrid_clock.cc:648] HybridClock initialized: now 1786232676725274 us; error 51 us; skew 500 ppm
I20260808 23:44:36.725682 6521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:36 dist-test-slave-5k54 krb5kdc[6507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232676, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:36.728806 6521 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:44:36.730232 6521 webserver.cc:460] Webserver started at http://127.18.186.190:44631/ using document root <none> and password file <none>
I20260808 23:44:36.730901 6521 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:36.731047 6521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:36.731355 6521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:36.734344 6521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c"
format_stamp: "Formatted at 2026-08-08 23:44:36 on dist-test-slave-5k54"
server_key: "9282d11e26a98a508f087f3a06df4654"
server_key_iv: "7b12604b877fa9d40449d71d52e451ce"
server_key_version: "encryptionkey@0"
I20260808 23:44:36.735091 6521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c"
format_stamp: "Formatted at 2026-08-08 23:44:36 on dist-test-slave-5k54"
server_key: "9282d11e26a98a508f087f3a06df4654"
server_key_iv: "7b12604b877fa9d40449d71d52e451ce"
server_key_version: "encryptionkey@0"
I20260808 23:44:36.740231 6521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 23:44:36.743625 6536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:36.744877 6521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:44:36.745060 6521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c"
format_stamp: "Formatted at 2026-08-08 23:44:36 on dist-test-slave-5k54"
server_key: "9282d11e26a98a508f087f3a06df4654"
server_key_iv: "7b12604b877fa9d40449d71d52e451ce"
server_key_version: "encryptionkey@0"
I20260808 23:44:36.745257 6521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:36.761533 6521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:36.765589 6521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:36.765877 6521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:36.784334 6521 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42101
I20260808 23:44:36.784358 6591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42101 every 8 connection(s)
I20260808 23:44:36.785741 6521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:36.788801 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 6521
I20260808 23:44:36.789021 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:36.789470 19178 external_mini_cluster.cc:1428] Setting key b8a8fb340c83a07aa52255102cf56c7e
I20260808 23:44:36.789625 6592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:36.804039 6592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c: Bootstrap starting.
Aug 08 23:44:36 dist-test-slave-5k54 krb5kdc[6507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:36.807855 6592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:36.809266 6592 log.cc:824] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:36.812772 6592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c: No bootstrap required, opened a new log
I20260808 23:44:36.814378 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:36.791344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44420 (local address 127.18.186.190:42101)
0808 23:44:36.792223 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:36.792238 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:36.792292 (+ 54us) server_negotiation.cc:407] Connection header received
0808 23:44:36.794626 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:36.794654 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:36.795173 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:36.795624 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:36.797008 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:36.799008 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:36.800400 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:36.801505 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:36.801686 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:36.805777 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:36.805789 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:36.805810 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:36.805865 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:36.809886 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:36.810826 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:36.810835 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:36.810844 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:36.811074 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:36.811447 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:36.811453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:36.811456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:36.812178 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:36.812480 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:36.812799 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 23:44:36.813366 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":356,"threads_started":1}
I20260808 23:44:36.821017 6592 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42101 } }
I20260808 23:44:36.821404 6592 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:36.821537 6592 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc5f3abd3a749bc9bfc4ef2a83eca1c, State: Initialized, Role: FOLLOWER
I20260808 23:44:36.822103 6592 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42101 } }
I20260808 23:44:36.822345 6592 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:36.822515 6592 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:36.822690 6592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:36.825161 6592 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42101 } }
I20260808 23:44:36.825675 6592 leader_election.cc:302] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfc5f3abd3a749bc9bfc4ef2a83eca1c; no voters:
I20260808 23:44:36.826344 6592 leader_election.cc:288] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:36.826472 6597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:36.827390 6597 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [term 1 LEADER]: Becoming Leader. State: Replica: dfc5f3abd3a749bc9bfc4ef2a83eca1c, State: Running, Role: LEADER
I20260808 23:44:36.827944 6597 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42101 } }
I20260808 23:44:36.828616 6592 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:36.830896 6598 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42101 } } }
I20260808 23:44:36.831400 6598 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:36.830976 6599 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfc5f3abd3a749bc9bfc4ef2a83eca1c. Latest consensus state: current_term: 1 leader_uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc5f3abd3a749bc9bfc4ef2a83eca1c" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42101 } } }
I20260808 23:44:36.833385 6599 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:36.834146 6607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:36.839489 6607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:36.848445 6607 catalog_manager.cc:1347] Generated new cluster ID: 3ce8ac06b95941e2a0f08f0d1a8414c2
I20260808 23:44:36.848604 6607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:36.877452 6607 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:36.879487 6607 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:36.889854 6607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c: Generated new TSK 0
I20260808 23:44:36.890702 6607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:44:36.974500 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42101
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:38839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:37.315917 6620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:37.316196 6620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:37.316367 6620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:37.323616 6620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:37.323770 6620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:37.323971 6620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:44:37.334437 6620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:38839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:37.335877 6620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:37.337172 6620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:37.350225 6625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:37.350294 6628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:37.350255 6626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:37.351380 6620 server_base.cc:1034] running on GCE node
I20260808 23:44:37.352003 6620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:37.352900 6620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:37.354103 6620 hybrid_clock.cc:648] HybridClock initialized: now 1786232677354056 us; error 42 us; skew 500 ppm
I20260808 23:44:37.354414 6620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:37 dist-test-slave-5k54 krb5kdc[6507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232677, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:37.360556 6620 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:44:37.361896 6620 webserver.cc:460] Webserver started at http://127.18.186.129:42811/ using document root <none> and password file <none>
I20260808 23:44:37.362514 6620 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:37.362615 6620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:37.362860 6620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:37.365694 6620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "548145e3a5754ed3b758c5eaa958a829"
format_stamp: "Formatted at 2026-08-08 23:44:37 on dist-test-slave-5k54"
server_key: "350bdf12d3c16666e1b234bea4742224"
server_key_iv: "aa5375524d097513092a8570fd3f11a7"
server_key_version: "encryptionkey@0"
I20260808 23:44:37.366392 6620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "548145e3a5754ed3b758c5eaa958a829"
format_stamp: "Formatted at 2026-08-08 23:44:37 on dist-test-slave-5k54"
server_key: "350bdf12d3c16666e1b234bea4742224"
server_key_iv: "aa5375524d097513092a8570fd3f11a7"
server_key_version: "encryptionkey@0"
I20260808 23:44:37.371600 6620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 23:44:37.374954 6635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:37.376266 6620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:37.376451 6620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "548145e3a5754ed3b758c5eaa958a829"
format_stamp: "Formatted at 2026-08-08 23:44:37 on dist-test-slave-5k54"
server_key: "350bdf12d3c16666e1b234bea4742224"
server_key_iv: "aa5375524d097513092a8570fd3f11a7"
server_key_version: "encryptionkey@0"
I20260808 23:44:37.376657 6620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:37.388391 6620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:37.392776 6620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:37.393067 6620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:37.393983 6620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:37.395263 6620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:37.395376 6620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:37.395547 6620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:37.395622 6620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:37.427999 6620 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:44733
I20260808 23:44:37.428052 6756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:44733 every 8 connection(s)
I20260808 23:44:37.429388 6620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:44:37.438099 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 6620
I20260808 23:44:37.438298 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:44:37.438771 19178 external_mini_cluster.cc:1428] Setting key 1f21f538f9eb4c4ccb981e948e5e080e
Aug 08 23:44:37 dist-test-slave-5k54 krb5kdc[6507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232677, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:37.451700 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:37.432008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:48307 (local address 127.18.186.190:42101)
0808 23:44:37.432451 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:37.432456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:37.433433 (+ 977us) server_negotiation.cc:407] Connection header received
0808 23:44:37.435584 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:37.435591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:37.435695 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:37.435817 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:37.437339 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:37.438355 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:37.440298 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:37.440902 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:37.441044 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:37.444544 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:37.444552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:37.444560 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:37.444603 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:37.448187 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:37.449020 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:37.449028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:37.449034 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:37.449190 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:37.449646 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:37.449654 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:37.449660 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:37.450061 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:37.450213 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:37.450903 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 23:44:37.451158 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":216,"threads_started":1}
I20260808 23:44:37.452292 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:37.432572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42101 (local address 127.18.186.129:48307)
0808 23:44:37.433276 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:37.433314 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:37.435253 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:37.436060 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:37.436068 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:37.436574 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:37.437154 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:37.437172 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:37.438521 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:37.438525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:37.440164 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:37.440178 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:37.441118 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:37.441123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:37.441317 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:37.442027 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:37.442049 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:37.444300 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:37.448319 (+ 4019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:37.448325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:37.448336 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:37.448889 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:37.449298 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:37.449301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:37.449303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:37.449484 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:37.450193 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:37.450199 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:37.450515 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 23:44:37.450731 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 23:44:37.451104 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":294,"threads_started":1}
I20260808 23:44:37.453891 6757 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42101
I20260808 23:44:37.454262 6757 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:37.455075 6757 heartbeater.cc:511] Master 127.18.186.190:42101 requested a full tablet report, sending...
I20260808 23:44:37.457212 6553 ts_manager.cc:194] Registered new tserver with Master: 548145e3a5754ed3b758c5eaa958a829 (127.18.186.129:44733)
I20260808 23:44:37.458636 6553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:48307
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:44:37.505234 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42101
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:38839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:37.840895 6763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:37.841171 6763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:37.841338 6763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:37.848735 6763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:37.848860 6763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:37.849043 6763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:44:37.858793 6763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:38839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:37.860129 6763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:37.861325 6763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:37.874236 6769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:37.874491 6768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:37.874262 6771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:37.875270 6763 server_base.cc:1034] running on GCE node
I20260808 23:44:37.875909 6763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:37.876834 6763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:37.878067 6763 hybrid_clock.cc:648] HybridClock initialized: now 1786232677878026 us; error 62 us; skew 500 ppm
I20260808 23:44:37.878388 6763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:37 dist-test-slave-5k54 krb5kdc[6507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232677, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:37.881649 6763 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:44:37.883096 6763 webserver.cc:460] Webserver started at http://127.18.186.130:37033/ using document root <none> and password file <none>
I20260808 23:44:37.883809 6763 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:37.883919 6763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:37.884182 6763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:37.887012 6763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "1165445317254529a419cfb71d396115"
format_stamp: "Formatted at 2026-08-08 23:44:37 on dist-test-slave-5k54"
server_key: "6b007932cc965eca0d6aa2d36ff36c8c"
server_key_iv: "0011497c181810214222a915d1a1d0ab"
server_key_version: "encryptionkey@0"
I20260808 23:44:37.887799 6763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "1165445317254529a419cfb71d396115"
format_stamp: "Formatted at 2026-08-08 23:44:37 on dist-test-slave-5k54"
server_key: "6b007932cc965eca0d6aa2d36ff36c8c"
server_key_iv: "0011497c181810214222a915d1a1d0ab"
server_key_version: "encryptionkey@0"
I20260808 23:44:37.892884 6763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 23:44:37.896165 6780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:37.897575 6763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:44:37.897708 6763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "1165445317254529a419cfb71d396115"
format_stamp: "Formatted at 2026-08-08 23:44:37 on dist-test-slave-5k54"
server_key: "6b007932cc965eca0d6aa2d36ff36c8c"
server_key_iv: "0011497c181810214222a915d1a1d0ab"
server_key_version: "encryptionkey@0"
I20260808 23:44:37.897871 6763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:37.912106 6763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:37.916504 6763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:37.916741 6763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:37.917616 6763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:37.918828 6763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:37.918901 6763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:37.918984 6763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:37.919032 6763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:37.952843 6763 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:43947
I20260808 23:44:37.952951 6899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:43947 every 8 connection(s)
I20260808 23:44:37.954176 6763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:44:37.964191 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 6763
I20260808 23:44:37.964399 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:44:37.964879 19178 external_mini_cluster.cc:1428] Setting key 412a5318e6bc74e0274088f945d946a6
Aug 08 23:44:37 dist-test-slave-5k54 krb5kdc[6507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232677, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:37.974375 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:37.956800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:59821 (local address 127.18.186.190:42101)
0808 23:44:37.957287 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:37.957291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:37.958112 (+ 821us) server_negotiation.cc:407] Connection header received
0808 23:44:37.960022 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:37.960027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:37.960127 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:37.960251 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:37.961802 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:37.962544 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:37.964258 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:37.964799 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:37.964936 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:37.968662 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:37.968667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:37.968671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:37.968702 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:37.971341 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:37.972078 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:37.972083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:37.972085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:37.972214 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:37.972668 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:37.972672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:37.972674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:37.972929 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:37.973089 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:37.973745 (+ 656us) server_negotiation.cc:299] Negotiation successful
0808 23:44:37.974001 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":266,"threads_started":1}
I20260808 23:44:37.975294 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:37.957224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42101 (local address 127.18.186.130:59821)
0808 23:44:37.957934 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:37.957977 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:37.959747 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:37.960491 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:37.960498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:37.961037 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:37.961592 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:37.961609 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:37.962732 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:37.962736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:37.964115 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:37.964127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:37.965020 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:37.965028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:37.965270 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:37.965912 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:37.965948 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:37.968441 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:37.971504 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:37.971515 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:37.971530 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:37.971943 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:37.972332 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:37.972337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:37.972341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:37.972536 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:37.973062 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:37.973069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:37.973399 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 23:44:37.973648 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 23:44:37.974160 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":278,"threads_started":1}
I20260808 23:44:37.976827 6900 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42101
I20260808 23:44:37.977140 6900 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:37.977890 6900 heartbeater.cc:511] Master 127.18.186.190:42101 requested a full tablet report, sending...
I20260808 23:44:37.979387 6553 ts_manager.cc:194] Registered new tserver with Master: 1165445317254529a419cfb71d396115 (127.18.186.130:43947)
I20260808 23:44:37.980482 6553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:59821
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:44:38.043237 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42101
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:38839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:38.371117 6906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:38.371374 6906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:38.371557 6906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:38.378563 6906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:38.378654 6906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:38.378834 6906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:44:38.388444 6906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:38839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:38.389654 6906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:38.390959 6906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:38.404165 6915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:38.404256 6911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:38.404194 6912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:38.405025 6906 server_base.cc:1034] running on GCE node
I20260808 23:44:38.405674 6906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:38.406594 6906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:38.407812 6906 hybrid_clock.cc:648] HybridClock initialized: now 1786232678407768 us; error 40 us; skew 500 ppm
I20260808 23:44:38.408126 6906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:38 dist-test-slave-5k54 krb5kdc[6507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232678, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:38.411284 6906 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:44:38.412767 6906 webserver.cc:460] Webserver started at http://127.18.186.131:40197/ using document root <none> and password file <none>
I20260808 23:44:38.413415 6906 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:38.413515 6906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:38.413769 6906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:38.416636 6906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "da9c8099a2bd41a3a13370f7d4782285"
format_stamp: "Formatted at 2026-08-08 23:44:38 on dist-test-slave-5k54"
server_key: "2c5ab4be4cb4ad5d34c22697003074e3"
server_key_iv: "73dcb114c94f9aac4c20f3374f3c877e"
server_key_version: "encryptionkey@0"
I20260808 23:44:38.417348 6906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "da9c8099a2bd41a3a13370f7d4782285"
format_stamp: "Formatted at 2026-08-08 23:44:38 on dist-test-slave-5k54"
server_key: "2c5ab4be4cb4ad5d34c22697003074e3"
server_key_iv: "73dcb114c94f9aac4c20f3374f3c877e"
server_key_version: "encryptionkey@0"
I20260808 23:44:38.422327 6906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:44:38.425704 6923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:38.426954 6906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:44:38.427109 6906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "da9c8099a2bd41a3a13370f7d4782285"
format_stamp: "Formatted at 2026-08-08 23:44:38 on dist-test-slave-5k54"
server_key: "2c5ab4be4cb4ad5d34c22697003074e3"
server_key_iv: "73dcb114c94f9aac4c20f3374f3c877e"
server_key_version: "encryptionkey@0"
I20260808 23:44:38.427400 6906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:38.452934 6906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:38.457146 6906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:38.457448 6906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:38.458483 6906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:38.459897 6906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:38.459972 6906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:38.460127 6906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:38.460182 6906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:38.461424 6757 heartbeater.cc:503] Master 127.18.186.190:42101 was elected leader, sending a full tablet report...
I20260808 23:44:38.494638 6906 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:37835
I20260808 23:44:38.494683 7041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:37835 every 8 connection(s)
I20260808 23:44:38.496045 6906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:44:38.501168 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 6906
I20260808 23:44:38.501384 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:44:38.501897 19178 external_mini_cluster.cc:1428] Setting key 06709e94669e87771ee80cbd2a1a5ec9
Aug 08 23:44:38 dist-test-slave-5k54 krb5kdc[6507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232678, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:38.517302 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.498658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:49769 (local address 127.18.186.190:42101)
0808 23:44:38.499154 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.499159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.500183 (+ 1024us) server_negotiation.cc:407] Connection header received
0808 23:44:38.502362 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.502370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.502460 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.502555 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:38.504337 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.505358 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.507316 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.507789 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.507927 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.511259 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:38.511264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:38.511267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:38.511317 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:38.513792 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:38.514505 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:38.514513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:38.514517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:38.514644 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:38.515169 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:38.515177 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:38.515181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:38.515625 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:38.515852 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.516659 (+ 807us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.516936 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":260,"threads_started":1}
I20260808 23:44:38.518211 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.499210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42101 (local address 127.18.186.131:49769)
0808 23:44:38.500028 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:38.500066 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:38.502053 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:38.502809 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:38.502816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:38.503489 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:38.504160 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.504178 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.505503 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.505508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.507134 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.507147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.508027 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.508037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.508272 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.508904 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:38.508925 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:38.511052 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:38.513937 (+ 2885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:38.513945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:38.513961 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:38.514387 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:38.514757 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:38.514762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:38.514766 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:38.514957 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:38.515769 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:38.515777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:38.516209 (+ 432us) client_negotiation.cc:769] Sending connection context
0808 23:44:38.516538 (+ 329us) client_negotiation.cc:240] Negotiation successful
0808 23:44:38.517052 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":310,"threads_started":1}
I20260808 23:44:38.519609 7042 heartbeater.cc:347] Connected to a master server at 127.18.186.190:42101
I20260808 23:44:38.519883 7042 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:38.520531 7042 heartbeater.cc:511] Master 127.18.186.190:42101 requested a full tablet report, sending...
I20260808 23:44:38.521957 6553 ts_manager.cc:194] Registered new tserver with Master: da9c8099a2bd41a3a13370f7d4782285 (127.18.186.131:37835)
I20260808 23:44:38.522990 6553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:49769
I20260808 23:44:38.529587 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:44:38.548247 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.533980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.18.186.190:42101)
0808 23:44:38.534135 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.534140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.534735 (+ 595us) server_negotiation.cc:407] Connection header received
0808 23:44:38.534924 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.534929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.535001 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.535105 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:38.536583 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.537299 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.538660 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.539096 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.539242 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.542292 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:38.542298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:38.542301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:38.542335 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:38.544935 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:38.545720 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:38.545725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:38.545734 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:38.545851 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:38.546325 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:38.546330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:38.546333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:38.546681 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:38.546992 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.547699 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.547903 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:44:38.552635 6553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:44:38.559584 6553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:44:38.588290 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.572554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:37835 (local address 127.0.0.1:54634)
0808 23:44:38.573417 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:38.573483 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:38.573676 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:38.574851 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:38.574862 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:38.574904 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:38.577011 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.577030 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.581104 (+ 4074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.581112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.584853 (+ 3741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.584867 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.587059 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.587066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.587454 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.587513 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 23:44:38.587655 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 23:44:38.587860 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":84,"thread_start_us":348,"threads_started":1}
I20260808 23:44:38.588564 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.572876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.18.186.131:37835)
0808 23:44:38.574050 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.574060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.574102 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:44:38.574225 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.574236 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.574542 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.577280 (+ 2738us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:38.577640 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.580966 (+ 3326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.585035 (+ 4069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.586491 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.586676 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.586888 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.587966 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.588153 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980,"thread_start_us":2477,"threads_started":1}
I20260808 23:44:38.590849 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.573745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:43947 (local address 127.0.0.1:39624)
0808 23:44:38.580585 (+ 6840us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:38.580610 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:38.580755 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:38.582134 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:38.582142 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:38.582181 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:38.582865 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.582888 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.585435 (+ 2547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.585440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.587700 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.587714 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.589941 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.589951 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.590130 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.590155 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:44:38.590274 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 23:44:38.590416 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6669,"mutex_wait_us":39,"thread_start_us":6479,"threads_started":1}
I20260808 23:44:38.591531 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.574261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39624 (local address 127.18.186.130:43947)
0808 23:44:38.581545 (+ 7284us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.581553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.581579 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:44:38.581674 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.581681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.581924 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.582126 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:38.583394 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.585118 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.587873 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.589544 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.589720 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.589929 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.590332 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.591147 (+ 815us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7125,"thread_start_us":6993,"threads_started":1}
I20260808 23:44:38.591761 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.573866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:44733 (local address 127.0.0.1:40452)
0808 23:44:38.583199 (+ 9333us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:38.583218 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:38.583333 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:38.584277 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:38.584282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:38.584305 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:38.584707 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.584720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.586675 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.586681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.588900 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.588915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.590977 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.590986 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.591150 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.591184 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:44:38.591265 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 23:44:38.591360 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9221,"mutex_wait_us":20,"thread_start_us":9065,"threads_started":1}
I20260808 23:44:38.592344 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.574260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.18.186.129:44733)
0808 23:44:38.582748 (+ 8488us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.582755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.583474 (+ 719us) server_negotiation.cc:407] Connection header received
0808 23:44:38.583588 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.583597 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.583811 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.584034 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:38.584923 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.586481 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.589094 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.590521 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.590708 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.590817 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.592000 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.592124 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8316,"thread_start_us":8815,"threads_started":1}
I20260808 23:44:38.593871 6973 tablet_service.cc:1467] Processing CreateTablet for tablet 18db96664d8846c18ff3807e971b56e6 (DEFAULT_TABLE table=test-table [id=52074114704b470b84990a614380acb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:38.595577 6973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18db96664d8846c18ff3807e971b56e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:38.595115 6828 tablet_service.cc:1467] Processing CreateTablet for tablet 18db96664d8846c18ff3807e971b56e6 (DEFAULT_TABLE table=test-table [id=52074114704b470b84990a614380acb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:38.596529 6828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18db96664d8846c18ff3807e971b56e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:38.596138 6685 tablet_service.cc:1467] Processing CreateTablet for tablet 18db96664d8846c18ff3807e971b56e6 (DEFAULT_TABLE table=test-table [id=52074114704b470b84990a614380acb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:38.597445 6685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18db96664d8846c18ff3807e971b56e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:38.608703 7062 tablet_bootstrap.cc:492] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Bootstrap starting.
I20260808 23:44:38.612206 7063 tablet_bootstrap.cc:492] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Bootstrap starting.
I20260808 23:44:38.612370 7062 tablet_bootstrap.cc:654] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:38.613076 7064 tablet_bootstrap.cc:492] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Bootstrap starting.
I20260808 23:44:38.613938 7062 log.cc:824] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:38.616737 7063 tablet_bootstrap.cc:654] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:38.616919 7064 tablet_bootstrap.cc:654] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:38.617636 7062 tablet_bootstrap.cc:492] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: No bootstrap required, opened a new log
I20260808 23:44:38.617805 7063 log.cc:824] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:38.617947 7062 ts_tablet_manager.cc:1397] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 23:44:38.618260 7064 log.cc:824] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:38.620682 7063 tablet_bootstrap.cc:492] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: No bootstrap required, opened a new log
I20260808 23:44:38.620966 7063 ts_tablet_manager.cc:1397] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 23:44:38.622895 7064 tablet_bootstrap.cc:492] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: No bootstrap required, opened a new log
I20260808 23:44:38.623147 7064 ts_tablet_manager.cc:1397] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 23:44:38.626513 7062 raft_consensus.cc:348] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.627133 7062 raft_consensus.cc:374] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:38.627310 7062 raft_consensus.cc:729] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9c8099a2bd41a3a13370f7d4782285, State: Initialized, Role: FOLLOWER
I20260808 23:44:38.628059 7062 consensus_queue.cc:260] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.629423 7042 heartbeater.cc:503] Master 127.18.186.190:42101 was elected leader, sending a full tablet report...
I20260808 23:44:38.630996 7062 ts_tablet_manager.cc:1428] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 23:44:38.631839 7063 raft_consensus.cc:348] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.632357 7063 raft_consensus.cc:374] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:38.632476 7063 raft_consensus.cc:729] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1165445317254529a419cfb71d396115, State: Initialized, Role: FOLLOWER
I20260808 23:44:38.633106 7063 consensus_queue.cc:260] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.633893 7064 raft_consensus.cc:348] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.634387 6900 heartbeater.cc:503] Master 127.18.186.190:42101 was elected leader, sending a full tablet report...
I20260808 23:44:38.634405 7064 raft_consensus.cc:374] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:38.634517 7064 raft_consensus.cc:729] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 548145e3a5754ed3b758c5eaa958a829, State: Initialized, Role: FOLLOWER
I20260808 23:44:38.635025 7063 ts_tablet_manager.cc:1428] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260808 23:44:38.635296 7064 consensus_queue.cc:260] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.637236 7064 ts_tablet_manager.cc:1428] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
W20260808 23:44:38.681679 6758 tablet.cc:2369] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:44:38.706266 6901 tablet.cc:2369] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:44:38.747750 7043 tablet.cc:2369] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:38.787690 7068 raft_consensus.cc:482] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:44:38.788076 7068 raft_consensus.cc:504] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.789891 7068 leader_election.cc:288] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1165445317254529a419cfb71d396115 (127.18.186.130:43947), 548145e3a5754ed3b758c5eaa958a829 (127.18.186.129:44733)
I20260808 23:44:38.796729 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.790586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:45667 (local address 127.18.186.130:43947)
0808 23:44:38.790860 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.790867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.791302 (+ 435us) server_negotiation.cc:407] Connection header received
0808 23:44:38.791698 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.791703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.791784 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.791885 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:38.792460 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.793274 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.794995 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.795813 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.795993 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.796066 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.796330 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.796463 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:44:38.796856 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.790274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:43947 (local address 127.18.186.131:45667)
0808 23:44:38.791193 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:38.791211 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:38.791525 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:38.791909 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:38.791913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:38.791937 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:38.792285 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.792295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.793431 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.793437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.794828 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.794840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.796009 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.796015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.796173 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.796197 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:44:38.796302 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 23:44:38.796429 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":23,"thread_start_us":592,"threads_started":1}
I20260808 23:44:38.797309 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.790127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:44733 (local address 127.18.186.131:53493)
0808 23:44:38.790667 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:38.790686 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:38.790827 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:38.791260 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:38.791265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:38.791294 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:38.791550 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.791563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.792772 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.792777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.794822 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:38.794837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.796582 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.796590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:38.796783 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.796819 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:44:38.796903 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 23:44:38.797037 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":1}
I20260808 23:44:38.797659 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.790395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:53493 (local address 127.18.186.129:44733)
0808 23:44:38.790616 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.790622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.790919 (+ 297us) server_negotiation.cc:407] Connection header received
0808 23:44:38.791015 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.791022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.791106 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.791222 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:38.791728 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.792635 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.795026 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.796274 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.796418 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.796488 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.797280 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.797421 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 23:44:38.798079 6850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18db96664d8846c18ff3807e971b56e6" candidate_uuid: "da9c8099a2bd41a3a13370f7d4782285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1165445317254529a419cfb71d396115" is_pre_election: true
I20260808 23:44:38.798712 6850 raft_consensus.cc:2454] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da9c8099a2bd41a3a13370f7d4782285 in term 0.
I20260808 23:44:38.798663 6706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18db96664d8846c18ff3807e971b56e6" candidate_uuid: "da9c8099a2bd41a3a13370f7d4782285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "548145e3a5754ed3b758c5eaa958a829" is_pre_election: true
I20260808 23:44:38.799153 6706 raft_consensus.cc:2454] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da9c8099a2bd41a3a13370f7d4782285 in term 0.
I20260808 23:44:38.799494 6925 leader_election.cc:302] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1165445317254529a419cfb71d396115, da9c8099a2bd41a3a13370f7d4782285; no voters:
I20260808 23:44:38.799934 7068 raft_consensus.cc:2781] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:44:38.800088 7068 raft_consensus.cc:482] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:44:38.800195 7068 raft_consensus.cc:3037] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:38.802673 7068 raft_consensus.cc:504] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.803363 7068 leader_election.cc:288] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [CANDIDATE]: Term 1 election: Requested vote from peers 1165445317254529a419cfb71d396115 (127.18.186.130:43947), 548145e3a5754ed3b758c5eaa958a829 (127.18.186.129:44733)
I20260808 23:44:38.803761 6850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18db96664d8846c18ff3807e971b56e6" candidate_uuid: "da9c8099a2bd41a3a13370f7d4782285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1165445317254529a419cfb71d396115"
I20260808 23:44:38.803984 6850 raft_consensus.cc:3037] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:38.803903 6706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18db96664d8846c18ff3807e971b56e6" candidate_uuid: "da9c8099a2bd41a3a13370f7d4782285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "548145e3a5754ed3b758c5eaa958a829"
I20260808 23:44:38.804163 6706 raft_consensus.cc:3037] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:38.806190 6850 raft_consensus.cc:2454] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da9c8099a2bd41a3a13370f7d4782285 in term 1.
I20260808 23:44:38.806502 6706 raft_consensus.cc:2454] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da9c8099a2bd41a3a13370f7d4782285 in term 1.
I20260808 23:44:38.806612 6925 leader_election.cc:302] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1165445317254529a419cfb71d396115, da9c8099a2bd41a3a13370f7d4782285; no voters:
I20260808 23:44:38.806919 7068 raft_consensus.cc:2781] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:38.807366 7068 raft_consensus.cc:686] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 1 LEADER]: Becoming Leader. State: Replica: da9c8099a2bd41a3a13370f7d4782285, State: Running, Role: LEADER
I20260808 23:44:38.807853 7068 consensus_queue.cc:237] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } }
I20260808 23:44:38.812713 6552 catalog_manager.cc:5495] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 reported cstate change: term changed from 0 to 1, leader changed from <none> to da9c8099a2bd41a3a13370f7d4782285 (127.18.186.131). New cstate: current_term: 1 leader_uuid: "da9c8099a2bd41a3a13370f7d4782285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da9c8099a2bd41a3a13370f7d4782285" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 37835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 } health_report { overall_health: UNKNOWN } } }
I20260808 23:44:38.886147 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.880238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54638 (local address 127.18.186.131:37835)
0808 23:44:38.880488 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.880493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.880752 (+ 259us) server_negotiation.cc:407] Connection header received
0808 23:44:38.880919 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.880923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.881005 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.881144 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:44:38.881588 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.882372 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.883846 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.884263 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.884410 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.885029 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:44:38.885528 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:44:38.885730 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.885850 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.885902 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 23:44:38.895049 6850 raft_consensus.cc:1261] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 1 FOLLOWER]: Refusing update from remote peer da9c8099a2bd41a3a13370f7d4782285: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:38.895048 6706 raft_consensus.cc:1261] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 1 FOLLOWER]: Refusing update from remote peer da9c8099a2bd41a3a13370f7d4782285: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:38.896190 7072 consensus_queue.cc:1035] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1165445317254529a419cfb71d396115" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:38.896533 7068 consensus_queue.cc:1035] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [LEADER]: Connected to new peer: Peer: permanent_uuid: "548145e3a5754ed3b758c5eaa958a829" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 44733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:38.911551 7075 mvcc.cc:205] Tried to move back new op lower bound from 7316409052742971392 to 7316409052400742400. Current Snapshot: MvccSnapshot[applied={T|T < 7316409052742971392}]
I20260808 23:44:38.913543 7076 mvcc.cc:205] Tried to move back new op lower bound from 7316409052742971392 to 7316409052400742400. Current Snapshot: MvccSnapshot[applied={T|T < 7316409052742971392}]
I20260808 23:44:38.925772 6553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:38.926069 6553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:38.929544 6553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dfc5f3abd3a749bc9bfc4ef2a83eca1c: Sending DeleteTablet for 3 replicas of tablet 18db96664d8846c18ff3807e971b56e6
I20260808 23:44:38.930724 6685 tablet_service.cc:1514] Processing DeleteTablet for tablet 18db96664d8846c18ff3807e971b56e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:38 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:40452
I20260808 23:44:38.930815 6828 tablet_service.cc:1514] Processing DeleteTablet for tablet 18db96664d8846c18ff3807e971b56e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:38 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:39624
I20260808 23:44:38.931743 6973 tablet_service.cc:1514] Processing DeleteTablet for tablet 18db96664d8846c18ff3807e971b56e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:38 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:54634
I20260808 23:44:38.931802 7087 tablet_replica.cc:330] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: stopping tablet replica
I20260808 23:44:38.932643 7087 raft_consensus.cc:2229] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:38.933360 7087 raft_consensus.cc:2258] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:38.934684 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 6620
I20260808 23:44:38.943912 7087 ts_tablet_manager.cc:1905] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:38.944721 7088 tablet_replica.cc:330] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: stopping tablet replica
I20260808 23:44:38.945243 7088 raft_consensus.cc:2229] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:38.945853 7088 raft_consensus.cc:2258] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:38.947990 7089 tablet_replica.cc:330] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: stopping tablet replica
I20260808 23:44:38.948192 7088 ts_tablet_manager.cc:1905] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:38.948361 7089 raft_consensus.cc:2229] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:44:38.949087 7089 raft_consensus.cc:2258] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:38.950980 7089 ts_tablet_manager.cc:1905] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:38.954635 7087 ts_tablet_manager.cc:1918] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:38.954829 7087 log.cc:1196] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/18db96664d8846c18ff3807e971b56e6
I20260808 23:44:38.955494 7087 ts_tablet_manager.cc:1939] T 18db96664d8846c18ff3807e971b56e6 P 1165445317254529a419cfb71d396115: Deleting consensus metadata
I20260808 23:44:38.956135 7088 ts_tablet_manager.cc:1918] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:38.956275 7088 log.cc:1196] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/18db96664d8846c18ff3807e971b56e6
I20260808 23:44:38.956817 7088 ts_tablet_manager.cc:1939] T 18db96664d8846c18ff3807e971b56e6 P 548145e3a5754ed3b758c5eaa958a829: Deleting consensus metadata
I20260808 23:44:38.957653 6538 catalog_manager.cc:4841] TS 1165445317254529a419cfb71d396115 (127.18.186.130:43947): tablet 18db96664d8846c18ff3807e971b56e6 (table test-table [id=52074114704b470b84990a614380acb2]) successfully deleted
I20260808 23:44:38.957770 7089 ts_tablet_manager.cc:1918] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:38.957908 7089 log.cc:1196] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/18db96664d8846c18ff3807e971b56e6
I20260808 23:44:38.958316 6540 catalog_manager.cc:4841] TS 548145e3a5754ed3b758c5eaa958a829 (127.18.186.129:44733): tablet 18db96664d8846c18ff3807e971b56e6 (table test-table [id=52074114704b470b84990a614380acb2]) successfully deleted
I20260808 23:44:38.958396 7089 ts_tablet_manager.cc:1939] T 18db96664d8846c18ff3807e971b56e6 P da9c8099a2bd41a3a13370f7d4782285: Deleting consensus metadata
I20260808 23:44:38.959959 6538 catalog_manager.cc:4841] TS da9c8099a2bd41a3a13370f7d4782285 (127.18.186.131:37835): tablet 18db96664d8846c18ff3807e971b56e6 (table test-table [id=52074114704b470b84990a614380acb2]) successfully deleted
Aug 08 23:44:38 dist-test-slave-5k54 krb5kdc[6507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:44:38.975135 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:38.935855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40454 (local address 127.18.186.129:44733)
0808 23:44:38.936670 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:38.936677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:38.948832 (+ 12155us) server_negotiation.cc:407] Connection header received
0808 23:44:38.949021 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:38.949026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:38.949123 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:38.949236 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:38.951560 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.952710 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.961670 (+ 8960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:38.962341 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:38.962486 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:38.968015 (+ 5529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:38.968030 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:38.968047 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:38.968093 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:38.971919 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:38.972811 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:38.972820 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:38.972824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:38.972995 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:38.973460 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:38.973466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:38.973470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:38.973908 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:38.974192 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:38.974533 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 23:44:38.974771 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663}
I20260808 23:44:38.975826 6751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:39.099893 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 6620
I20260808 23:44:39.127679 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 6763
Aug 08 23:44:39 dist-test-slave-5k54 krb5kdc[6507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:44:39.146127 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:39.128522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.18.186.130:43947)
0808 23:44:39.128710 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:39.128716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:39.128824 (+ 108us) server_negotiation.cc:407] Connection header received
0808 23:44:39.129010 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:39.129013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:39.129121 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:39.129217 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:39.131030 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:39.131931 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:39.133862 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:39.134312 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:39.134423 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:39.139978 (+ 5555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:39.139993 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:39.140008 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:39.140050 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:39.142984 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:39.143830 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:39.143837 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:39.143839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:39.143973 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:39.144474 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:39.144479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:39.144481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:39.144944 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:39.145191 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:39.145548 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 23:44:39.145716 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:44:39.146844 6894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:39.247275 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 6763
I20260808 23:44:39.273227 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 6906
Aug 08 23:44:39 dist-test-slave-5k54 krb5kdc[6507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:44:39.292001 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:39.274030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.18.186.131:37835)
0808 23:44:39.274206 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:39.274210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:39.274298 (+ 88us) server_negotiation.cc:407] Connection header received
0808 23:44:39.274511 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:39.274515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:39.274578 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:39.274665 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:39.276576 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:39.277401 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:39.279250 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:39.279911 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:39.280006 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:39.285882 (+ 5876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:39.285894 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:39.285909 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:39.285942 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:39.289053 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:39.289924 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:39.289929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:39.289932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:39.290080 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:39.290504 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:39.290507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:39.290508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:39.290846 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:39.291122 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:39.291527 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 23:44:39.291696 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:44:39.293035 7036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:39.414057 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 6906
I20260808 23:44:39.436159 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 6521
I20260808 23:44:39.437227 6585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:39.550841 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 6521
2026-08-08T23:44:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4656 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T23:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:39Z Disabled control of system clock
I20260808 23:44:39.641942 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:41639
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:37151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:44:39.979228 7098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:39.979513 7098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:39.979596 7098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:39.986653 7098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:39.986753 7098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:39.986799 7098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:39.986858 7098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:39.986910 7098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:39.995784 7098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:41639
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:41639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:39.996999 7098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:39.998687 7098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:40.010800 7104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:40.010875 7103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:40.010923 7106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:40.011793 7098 server_base.cc:1034] running on GCE node
I20260808 23:44:40.012653 7098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:40.014133 7098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:40.015403 7098 hybrid_clock.cc:648] HybridClock initialized: now 1786232680015360 us; error 55 us; skew 500 ppm
I20260808 23:44:40.015810 7098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:40.018220 7098 webserver.cc:460] Webserver started at http://127.18.186.190:33867/ using document root <none> and password file <none>
I20260808 23:44:40.018931 7098 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:40.019070 7098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:40.019475 7098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:40.022542 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "887d84bfee5c4912bc74411e0ca0c259"
format_stamp: "Formatted at 2026-08-08 23:44:40 on dist-test-slave-5k54"
server_key: "191335cb9ec4657a2dc6c437f445269e"
server_key_iv: "e0b57d489ecba6805f81ed34a38d8a6a"
server_key_version: "encryptionkey@0"
I20260808 23:44:40.023370 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "887d84bfee5c4912bc74411e0ca0c259"
format_stamp: "Formatted at 2026-08-08 23:44:40 on dist-test-slave-5k54"
server_key: "191335cb9ec4657a2dc6c437f445269e"
server_key_iv: "e0b57d489ecba6805f81ed34a38d8a6a"
server_key_version: "encryptionkey@0"
I20260808 23:44:40.028620 7098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 23:44:40.032217 7112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:40.033512 7098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:44:40.033711 7098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "887d84bfee5c4912bc74411e0ca0c259"
format_stamp: "Formatted at 2026-08-08 23:44:40 on dist-test-slave-5k54"
server_key: "191335cb9ec4657a2dc6c437f445269e"
server_key_iv: "e0b57d489ecba6805f81ed34a38d8a6a"
server_key_version: "encryptionkey@0"
I20260808 23:44:40.033938 7098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:40.059022 7098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:40.059968 7098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:40.060288 7098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:40.079686 7098 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:41639
I20260808 23:44:40.079715 7167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:41639 every 8 connection(s)
I20260808 23:44:40.081138 7098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:40.085034 7168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:40.087533 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 7098
I20260808 23:44:40.087721 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:40.088277 19178 external_mini_cluster.cc:1428] Setting key 33391fe1b4ee4f5007ecee1dde6f0cb4
I20260808 23:44:40.097115 7168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259: Bootstrap starting.
I20260808 23:44:40.100392 7168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:40.101583 7168 log.cc:824] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:40.102125 7171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:40.090379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.18.186.190:41639)
0808 23:44:40.091573 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:40.091592 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:40.091637 (+ 45us) server_negotiation.cc:407] Connection header received
0808 23:44:40.093346 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:40.093368 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:40.093812 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:40.094230 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:40.094814 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:40.096170 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:40.097581 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:40.098295 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:40.098523 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:40.099773 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:40.099781 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:40.099796 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:40.100333 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:40.100428 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:40.100449 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:40.100575 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:40.100798 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 23:44:40.101109 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":413,"threads_started":1}
I20260808 23:44:40.104921 7168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259: No bootstrap required, opened a new log
I20260808 23:44:40.112520 7168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d84bfee5c4912bc74411e0ca0c259" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41639 } }
I20260808 23:44:40.112907 7168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:40.113018 7168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887d84bfee5c4912bc74411e0ca0c259, State: Initialized, Role: FOLLOWER
I20260808 23:44:40.113627 7168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d84bfee5c4912bc74411e0ca0c259" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41639 } }
I20260808 23:44:40.113890 7168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:40.114043 7168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:40.114244 7168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:40.116528 7168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d84bfee5c4912bc74411e0ca0c259" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41639 } }
I20260808 23:44:40.117043 7168 leader_election.cc:302] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 887d84bfee5c4912bc74411e0ca0c259; no voters:
I20260808 23:44:40.117630 7168 leader_election.cc:288] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:40.117786 7174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:40.118947 7174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [term 1 LEADER]: Becoming Leader. State: Replica: 887d84bfee5c4912bc74411e0ca0c259, State: Running, Role: LEADER
I20260808 23:44:40.119567 7174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d84bfee5c4912bc74411e0ca0c259" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41639 } }
I20260808 23:44:40.120002 7168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:40.122965 7178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 887d84bfee5c4912bc74411e0ca0c259. Latest consensus state: current_term: 1 leader_uuid: "887d84bfee5c4912bc74411e0ca0c259" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d84bfee5c4912bc74411e0ca0c259" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41639 } } }
I20260808 23:44:40.123052 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "887d84bfee5c4912bc74411e0ca0c259" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d84bfee5c4912bc74411e0ca0c259" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 41639 } } }
I20260808 23:44:40.123461 7178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:40.123459 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:40.125537 7186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:40.130959 7186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:40.140060 7186 catalog_manager.cc:1347] Generated new cluster ID: 09898f488bf944e98fdd0234f7559d01
I20260808 23:44:40.140193 7186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:40.149737 7186 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:40.150919 7186 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:40.156919 7186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259: Generated new TSK 0
I20260808 23:44:40.157883 7186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:44:40.169232 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41639
--builtin_ntp_servers=127.18.186.148:37151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:44:40.508980 7197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:40.509232 7197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:40.509392 7197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:40.516670 7197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:40.516768 7197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:40.516927 7197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:44:40.526387 7197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:40.527660 7197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:40.529374 7197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:40.542640 7202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:40.542691 7203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:40.542594 7205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:40.544118 7197 server_base.cc:1034] running on GCE node
I20260808 23:44:40.544713 7197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:40.545655 7197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:40.546888 7197 hybrid_clock.cc:648] HybridClock initialized: now 1786232680546854 us; error 41 us; skew 500 ppm
I20260808 23:44:40.547214 7197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:40.549461 7197 webserver.cc:460] Webserver started at http://127.18.186.129:34897/ using document root <none> and password file <none>
I20260808 23:44:40.550146 7197 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:40.550282 7197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:40.550611 7197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:40.553499 7197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "57fe64c21a494d5b84a8d0382bde526f"
format_stamp: "Formatted at 2026-08-08 23:44:40 on dist-test-slave-5k54"
server_key: "b7980a224ff34a6a690b8151e49653b0"
server_key_iv: "a06afc1cd448e06c2165d075a70b53cb"
server_key_version: "encryptionkey@0"
I20260808 23:44:40.554283 7197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "57fe64c21a494d5b84a8d0382bde526f"
format_stamp: "Formatted at 2026-08-08 23:44:40 on dist-test-slave-5k54"
server_key: "b7980a224ff34a6a690b8151e49653b0"
server_key_iv: "a06afc1cd448e06c2165d075a70b53cb"
server_key_version: "encryptionkey@0"
I20260808 23:44:40.559185 7197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 23:44:40.562652 7211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:40.564090 7197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:40.564261 7197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "57fe64c21a494d5b84a8d0382bde526f"
format_stamp: "Formatted at 2026-08-08 23:44:40 on dist-test-slave-5k54"
server_key: "b7980a224ff34a6a690b8151e49653b0"
server_key_iv: "a06afc1cd448e06c2165d075a70b53cb"
server_key_version: "encryptionkey@0"
I20260808 23:44:40.564496 7197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:40.579293 7197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:40.580102 7197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:40.580431 7197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:40.581372 7197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:40.582718 7197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:40.582841 7197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:40.582979 7197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:40.583069 7197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:40.616786 7197 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:45933
I20260808 23:44:40.616876 7336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:45933 every 8 connection(s)
I20260808 23:44:40.618101 7197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:44:40.620463 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 7197
I20260808 23:44:40.620659 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:44:40.621230 19178 external_mini_cluster.cc:1428] Setting key 9db2200865d960404321ab7bcebc799a
I20260808 23:44:40.625982 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41639
--builtin_ntp_servers=127.18.186.148:37151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:44:40.632707 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:40.620697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:33579 (local address 127.18.186.190:41639)
0808 23:44:40.623832 (+ 3135us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:40.623837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:40.623870 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:44:40.624850 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:40.624856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:40.624960 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:40.625087 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:40.626264 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:40.626973 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:40.629505 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:40.629908 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:40.630001 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:40.631638 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:40.631642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:40.631645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:40.631662 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:40.631722 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:40.631735 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:40.631889 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:40.632267 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 23:44:40.632361 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3021,"thread_start_us":2918,"threads_started":1}
I20260808 23:44:40.633682 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:40.621605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41639 (local address 127.18.186.129:33579)
0808 23:44:40.622649 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:40.622684 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:40.624497 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:40.625347 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:40.625366 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:40.625455 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:40.626063 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:40.626080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:40.627136 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:40.627144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:40.629321 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:40.629338 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:40.630123 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:40.630132 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:40.630482 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:40.631262 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:40.631267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:40.631320 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:40.631448 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:40.631891 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:40.631904 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:40.631915 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 23:44:40.632240 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 23:44:40.632615 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":361,"threads_started":1}
I20260808 23:44:40.635211 7337 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41639
I20260808 23:44:40.635564 7337 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:40.636333 7337 heartbeater.cc:511] Master 127.18.186.190:41639 requested a full tablet report, sending...
I20260808 23:44:40.638623 7130 ts_manager.cc:194] Registered new tserver with Master: 57fe64c21a494d5b84a8d0382bde526f (127.18.186.129:45933)
I20260808 23:44:40.640185 7130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.129:33579
W20260808 23:44:40.968799 7342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:40.969069 7342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:40.969244 7342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:40.976325 7342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:40.976415 7342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:40.976574 7342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:44:40.985350 7342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:40.986567 7342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:40.988361 7342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:41.001632 7347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:41.001662 7348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:41.001811 7351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:41.002568 7342 server_base.cc:1034] running on GCE node
I20260808 23:44:41.003216 7342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:41.004212 7342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:41.005461 7342 hybrid_clock.cc:648] HybridClock initialized: now 1786232681005405 us; error 67 us; skew 500 ppm
I20260808 23:44:41.005802 7342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:41.008211 7342 webserver.cc:460] Webserver started at http://127.18.186.130:39287/ using document root <none> and password file <none>
I20260808 23:44:41.008944 7342 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:41.009089 7342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:41.009426 7342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:41.012373 7342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a"
format_stamp: "Formatted at 2026-08-08 23:44:41 on dist-test-slave-5k54"
server_key: "1ca331f814462960fbb80b8c93be11e3"
server_key_iv: "b15d82d0289418c2a03beda144a054b0"
server_key_version: "encryptionkey@0"
I20260808 23:44:41.013147 7342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a"
format_stamp: "Formatted at 2026-08-08 23:44:41 on dist-test-slave-5k54"
server_key: "1ca331f814462960fbb80b8c93be11e3"
server_key_iv: "b15d82d0289418c2a03beda144a054b0"
server_key_version: "encryptionkey@0"
I20260808 23:44:41.018399 7342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:44:41.021975 7357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:41.023361 7342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:44:41.023663 7342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a"
format_stamp: "Formatted at 2026-08-08 23:44:41 on dist-test-slave-5k54"
server_key: "1ca331f814462960fbb80b8c93be11e3"
server_key_iv: "b15d82d0289418c2a03beda144a054b0"
server_key_version: "encryptionkey@0"
I20260808 23:44:41.023880 7342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:41.035303 7342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:41.036095 7342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:41.036402 7342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:41.037374 7342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:41.039042 7342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:41.039158 7342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:41.039296 7342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:41.039393 7342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:41.071281 7342 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:36465
I20260808 23:44:41.071337 7480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:36465 every 8 connection(s)
I20260808 23:44:41.072651 7342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:44:41.079321 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 7342
I20260808 23:44:41.079555 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:44:41.080027 19178 external_mini_cluster.cc:1428] Setting key 36891bd23e6c034ad19221a6b9943bc9
I20260808 23:44:41.083006 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41639
--builtin_ntp_servers=127.18.186.148:37151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:44:41.086862 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.075235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:38135 (local address 127.18.186.190:41639)
0808 23:44:41.075500 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.075506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.076923 (+ 1417us) server_negotiation.cc:407] Connection header received
0808 23:44:41.078651 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.078655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.078741 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.078826 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:41.080226 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.081120 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.083449 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.084115 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.084213 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.085994 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:41.086001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:41.086006 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:41.086026 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:41.086080 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:41.086095 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:41.086150 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.086460 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.086555 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 23:44:41.088739 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.075851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41639 (local address 127.18.186.130:38135)
0808 23:44:41.076748 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:41.076789 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:41.078416 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:41.079129 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:41.079141 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:41.079259 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:41.080051 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.080073 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.081337 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.081342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.083254 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.083269 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.084506 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.084513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.084814 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.085655 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:41.085659 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:41.085706 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:41.085796 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:41.086230 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:41.086241 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:41.086249 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:44:41.087100 (+ 851us) client_negotiation.cc:240] Negotiation successful
0808 23:44:41.087514 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":282,"threads_started":1}
I20260808 23:44:41.090353 7481 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41639
I20260808 23:44:41.090653 7481 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:41.091452 7481 heartbeater.cc:511] Master 127.18.186.190:41639 requested a full tablet report, sending...
I20260808 23:44:41.093060 7130 ts_manager.cc:194] Registered new tserver with Master: 9438d7bf75cd404a9c530e2d8cd6cd3a (127.18.186.130:36465)
I20260808 23:44:41.094276 7130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.130:38135
W20260808 23:44:41.416632 7485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:41.416900 7485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:41.417050 7485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:41.424600 7485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:41.424701 7485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:41.424911 7485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:44:41.434353 7485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:37151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:41.435746 7485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:41.437605 7485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:41.450649 7490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:41.450736 7491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:41.450704 7494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:41.451994 7485 server_base.cc:1034] running on GCE node
I20260808 23:44:41.452548 7485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:41.453513 7485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:41.454715 7485 hybrid_clock.cc:648] HybridClock initialized: now 1786232681454667 us; error 45 us; skew 500 ppm
I20260808 23:44:41.455006 7485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:41.457334 7485 webserver.cc:460] Webserver started at http://127.18.186.131:35299/ using document root <none> and password file <none>
I20260808 23:44:41.457993 7485 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:41.458079 7485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:41.458335 7485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:41.461158 7485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "4453151643db4593b58971413bed31ee"
format_stamp: "Formatted at 2026-08-08 23:44:41 on dist-test-slave-5k54"
server_key: "86ebce10e83994e476c91fb984cb2ee1"
server_key_iv: "e7b0ec9566596abc2dee6938eae0da15"
server_key_version: "encryptionkey@0"
I20260808 23:44:41.461895 7485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "4453151643db4593b58971413bed31ee"
format_stamp: "Formatted at 2026-08-08 23:44:41 on dist-test-slave-5k54"
server_key: "86ebce10e83994e476c91fb984cb2ee1"
server_key_iv: "e7b0ec9566596abc2dee6938eae0da15"
server_key_version: "encryptionkey@0"
I20260808 23:44:41.466964 7485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 23:44:41.470379 7500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:41.471772 7485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 23:44:41.471940 7485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "4453151643db4593b58971413bed31ee"
format_stamp: "Formatted at 2026-08-08 23:44:41 on dist-test-slave-5k54"
server_key: "86ebce10e83994e476c91fb984cb2ee1"
server_key_iv: "e7b0ec9566596abc2dee6938eae0da15"
server_key_version: "encryptionkey@0"
I20260808 23:44:41.472159 7485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:41.489871 7485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:41.490640 7485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:41.490942 7485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:41.492004 7485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:41.493387 7485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:41.493520 7485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:41.493605 7485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:41.493670 7485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:41.527799 7485 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:40371
I20260808 23:44:41.527822 7624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:40371 every 8 connection(s)
I20260808 23:44:41.529284 7485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:44:41.534372 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 7485
I20260808 23:44:41.534572 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:44:41.535039 19178 external_mini_cluster.cc:1428] Setting key acc1e43ac213bece5ce33593aee104cb
I20260808 23:44:41.542248 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.531770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:47961 (local address 127.18.186.190:41639)
0808 23:44:41.531956 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.531960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.533288 (+ 1328us) server_negotiation.cc:407] Connection header received
0808 23:44:41.535322 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.535329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.535499 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.535612 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:41.537113 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.538162 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.539941 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.540314 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.540418 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.541611 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:41.541615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:41.541618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:41.541634 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:41.541661 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:41.541672 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:41.541755 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.541945 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.542019 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:44:41.543205 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.532262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:41639 (local address 127.18.186.131:47961)
0808 23:44:41.533138 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:41.533173 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:41.535013 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:41.535923 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:41.535935 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:41.536065 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:41.536911 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.536936 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.538288 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.538292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.539819 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.539831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.540490 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.540495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.540674 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.541348 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:41.541351 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:41.541389 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:41.541467 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:41.541771 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:41.541779 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:41.541784 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:44:41.541918 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 23:44:41.542200 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":257,"threads_started":1}
I20260808 23:44:41.544492 7625 heartbeater.cc:347] Connected to a master server at 127.18.186.190:41639
I20260808 23:44:41.544736 7625 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:41.545423 7625 heartbeater.cc:511] Master 127.18.186.190:41639 requested a full tablet report, sending...
I20260808 23:44:41.546653 7130 ts_manager.cc:194] Registered new tserver with Master: 4453151643db4593b58971413bed31ee (127.18.186.131:40371)
I20260808 23:44:41.547396 7130 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.131:47961
I20260808 23:44:41.550467 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:44:41.561057 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.554560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.18.186.190:41639)
0808 23:44:41.554728 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.554732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.555244 (+ 512us) server_negotiation.cc:407] Connection header received
0808 23:44:41.555365 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.555368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.555494 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.555619 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:41.556103 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.556779 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.558147 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.558844 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.559007 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.560225 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:41.560229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:41.560232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:41.560247 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:41.560275 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:41.560285 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:41.560437 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.560773 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.560819 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:44:41.565248 7130 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:44:41.572227 7130 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:44:41.610074 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.584477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60844 (local address 127.18.186.129:45933)
0808 23:44:41.600733 (+ 16256us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.600746 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.600770 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:44:41.600860 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.600866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.601061 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.601284 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:41.603273 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.604855 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.607668 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.608775 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.608961 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.609185 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.609465 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.609637 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16117,"thread_start_us":15995,"threads_started":1}
I20260808 23:44:41.610723 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.584121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:36465 (local address 127.0.0.1:39296)
0808 23:44:41.600520 (+ 16399us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:41.600540 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:41.601334 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:41.602278 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:41.602286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:41.602315 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:41.602978 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.603002 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.604606 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.604613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.607303 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.607320 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.609900 (+ 2580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.609905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.610104 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.610134 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 23:44:41.610224 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:44:41.610334 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16280,"mutex_wait_us":235,"thread_start_us":16097,"threads_started":1}
I20260808 23:44:41.610934 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.584297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39296 (local address 127.18.186.130:36465)
0808 23:44:41.600275 (+ 15978us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.600289 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.600638 (+ 349us) server_negotiation.cc:407] Connection header received
0808 23:44:41.601621 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.601632 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.601851 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.602064 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:41.603191 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.604475 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.607539 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.609370 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.609561 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.609778 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.610452 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.610619 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15764,"thread_start_us":15630,"threads_started":1}
I20260808 23:44:41.610775 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.584098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60844)
0808 23:44:41.599197 (+ 15099us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:41.599254 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:41.599530 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:41.602401 (+ 2871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:41.602412 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:41.602434 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:41.603036 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.603052 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.605001 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.605006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.607480 (+ 2474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.607496 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.608965 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.608970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.609163 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.609216 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 23:44:41.610280 (+ 1064us) client_negotiation.cc:240] Negotiation successful
0808 23:44:41.610415 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14696,"mutex_wait_us":193,"thread_start_us":16547,"threads_started":1}
I20260808 23:44:41.611622 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.601512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.18.186.131:40371)
0808 23:44:41.602718 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.602727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.602761 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:44:41.602862 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.602870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.603099 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.603286 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:41.603903 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.605365 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.608040 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.609456 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.610757 (+ 1301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.610975 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.611046 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.611216 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1031,"thread_start_us":856,"threads_started":1}
I20260808 23:44:41.611860 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.601130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:40371 (local address 127.0.0.1:57906)
0808 23:44:41.601938 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:41.601973 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:41.602118 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:41.603272 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:41.603278 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:41.603306 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:41.603715 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.603727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.605552 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.605562 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.607795 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.607812 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.609668 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.609674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.609860 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.609903 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 23:44:41.609976 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:44:41.610274 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":126,"thread_start_us":511,"threads_started":1}
I20260808 23:44:41.615080 7266 tablet_service.cc:1467] Processing CreateTablet for tablet b2d0b09962b5432a815439e10f966c17 (DEFAULT_TABLE table=test-table [id=67d38266fcb24dcd98cdf55f0ce17ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:41.615180 7412 tablet_service.cc:1467] Processing CreateTablet for tablet b2d0b09962b5432a815439e10f966c17 (DEFAULT_TABLE table=test-table [id=67d38266fcb24dcd98cdf55f0ce17ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:41.615181 7553 tablet_service.cc:1467] Processing CreateTablet for tablet b2d0b09962b5432a815439e10f966c17 (DEFAULT_TABLE table=test-table [id=67d38266fcb24dcd98cdf55f0ce17ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:41.616582 7553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d0b09962b5432a815439e10f966c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:41.616585 7412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d0b09962b5432a815439e10f966c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:41.620857 7266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d0b09962b5432a815439e10f966c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:41.631734 7644 tablet_bootstrap.cc:492] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: Bootstrap starting.
I20260808 23:44:41.634302 7643 tablet_bootstrap.cc:492] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Bootstrap starting.
I20260808 23:44:41.635516 7645 tablet_bootstrap.cc:492] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: Bootstrap starting.
I20260808 23:44:41.637657 7643 tablet_bootstrap.cc:654] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:41.638499 7645 tablet_bootstrap.cc:654] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:41.638985 7643 log.cc:824] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:41.639845 7645 log.cc:824] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:41.640153 7644 tablet_bootstrap.cc:654] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:41.641377 7644 log.cc:824] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:41.642395 7643 tablet_bootstrap.cc:492] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: No bootstrap required, opened a new log
I20260808 23:44:41.642681 7643 ts_tablet_manager.cc:1397] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 23:44:41.643007 7337 heartbeater.cc:503] Master 127.18.186.190:41639 was elected leader, sending a full tablet report...
I20260808 23:44:41.646070 7644 tablet_bootstrap.cc:492] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: No bootstrap required, opened a new log
I20260808 23:44:41.646106 7645 tablet_bootstrap.cc:492] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: No bootstrap required, opened a new log
I20260808 23:44:41.646406 7645 ts_tablet_manager.cc:1397] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260808 23:44:41.646457 7644 ts_tablet_manager.cc:1397] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.004s
I20260808 23:44:41.653363 7643 raft_consensus.cc:348] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.653957 7643 raft_consensus.cc:374] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:41.654100 7643 raft_consensus.cc:729] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4453151643db4593b58971413bed31ee, State: Initialized, Role: FOLLOWER
I20260808 23:44:41.654058 7644 raft_consensus.cc:348] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.654610 7644 raft_consensus.cc:374] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:41.654739 7644 raft_consensus.cc:729] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9438d7bf75cd404a9c530e2d8cd6cd3a, State: Initialized, Role: FOLLOWER
I20260808 23:44:41.654805 7643 consensus_queue.cc:260] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.655309 7644 consensus_queue.cc:260] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.656193 7625 heartbeater.cc:503] Master 127.18.186.190:41639 was elected leader, sending a full tablet report...
I20260808 23:44:41.656744 7643 ts_tablet_manager.cc:1428] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 23:44:41.657218 7645 raft_consensus.cc:348] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.657706 7645 raft_consensus.cc:374] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:41.657779 7481 heartbeater.cc:503] Master 127.18.186.190:41639 was elected leader, sending a full tablet report...
I20260808 23:44:41.657864 7645 raft_consensus.cc:729] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57fe64c21a494d5b84a8d0382bde526f, State: Initialized, Role: FOLLOWER
I20260808 23:44:41.658442 7645 consensus_queue.cc:260] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.658663 7644 ts_tablet_manager.cc:1428] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260808 23:44:41.660447 7645 ts_tablet_manager.cc:1428] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260808 23:44:41.702385 7649 raft_consensus.cc:482] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:44:41.702744 7649 raft_consensus.cc:504] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.704555 7649 leader_election.cc:288] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9438d7bf75cd404a9c530e2d8cd6cd3a (127.18.186.130:36465), 57fe64c21a494d5b84a8d0382bde526f (127.18.186.129:45933)
I20260808 23:44:41.710986 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.704847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:36465 (local address 127.18.186.131:35185)
0808 23:44:41.705044 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:41.705066 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:41.705202 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:41.705660 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:41.705664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:41.705694 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:41.705943 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.705952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.707299 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.707303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.709159 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.709174 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.710464 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.710468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.710580 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.710605 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:44:41.710675 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:44:41.710769 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 23:44:41.711163 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.704990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:35185 (local address 127.18.186.130:36465)
0808 23:44:41.705218 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.705227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.705302 (+ 75us) server_negotiation.cc:407] Connection header received
0808 23:44:41.705400 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.705410 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.705511 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.705615 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:41.706144 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.707169 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.709352 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.710316 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.710482 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.710569 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.710733 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.710872 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:44:41.712380 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.705021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.18.186.131:37261)
0808 23:44:41.705797 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:41.705814 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:41.705934 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:41.706410 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:41.706413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:41.706476 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:41.706832 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.706845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.708110 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.708116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.709867 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:41.709878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.711649 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.711655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:41.711833 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.711855 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 23:44:41.711968 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 23:44:41.712090 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":407,"threads_started":1}
I20260808 23:44:41.712383 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.705197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:37261 (local address 127.18.186.129:45933)
0808 23:44:41.705432 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.705441 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.705911 (+ 470us) server_negotiation.cc:407] Connection header received
0808 23:44:41.706160 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.706167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.706271 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.706401 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:41.707033 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.707947 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.710067 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.711474 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.711612 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.711660 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.711996 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.712122 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 23:44:41.712244 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d0b09962b5432a815439e10f966c17" candidate_uuid: "4453151643db4593b58971413bed31ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" is_pre_election: true
I20260808 23:44:41.712869 7434 raft_consensus.cc:2454] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4453151643db4593b58971413bed31ee in term 0.
I20260808 23:44:41.713641 7503 leader_election.cc:302] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4453151643db4593b58971413bed31ee, 9438d7bf75cd404a9c530e2d8cd6cd3a; no voters:
I20260808 23:44:41.713570 7288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d0b09962b5432a815439e10f966c17" candidate_uuid: "4453151643db4593b58971413bed31ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57fe64c21a494d5b84a8d0382bde526f" is_pre_election: true
I20260808 23:44:41.714051 7649 raft_consensus.cc:2781] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:44:41.714128 7288 raft_consensus.cc:2454] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4453151643db4593b58971413bed31ee in term 0.
I20260808 23:44:41.714214 7649 raft_consensus.cc:482] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:44:41.714320 7649 raft_consensus.cc:3037] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:41.716742 7649 raft_consensus.cc:504] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.717291 7649 leader_election.cc:288] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [CANDIDATE]: Term 1 election: Requested vote from peers 9438d7bf75cd404a9c530e2d8cd6cd3a (127.18.186.130:36465), 57fe64c21a494d5b84a8d0382bde526f (127.18.186.129:45933)
I20260808 23:44:41.717752 7288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d0b09962b5432a815439e10f966c17" candidate_uuid: "4453151643db4593b58971413bed31ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57fe64c21a494d5b84a8d0382bde526f"
I20260808 23:44:41.717752 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d0b09962b5432a815439e10f966c17" candidate_uuid: "4453151643db4593b58971413bed31ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a"
I20260808 23:44:41.717957 7434 raft_consensus.cc:3037] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:41.717970 7288 raft_consensus.cc:3037] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:41.721268 7434 raft_consensus.cc:2454] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4453151643db4593b58971413bed31ee in term 1.
I20260808 23:44:41.721268 7288 raft_consensus.cc:2454] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4453151643db4593b58971413bed31ee in term 1.
I20260808 23:44:41.721716 7504 leader_election.cc:302] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4453151643db4593b58971413bed31ee, 57fe64c21a494d5b84a8d0382bde526f; no voters:
I20260808 23:44:41.722095 7649 raft_consensus.cc:2781] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:41.722559 7649 raft_consensus.cc:686] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 1 LEADER]: Becoming Leader. State: Replica: 4453151643db4593b58971413bed31ee, State: Running, Role: LEADER
I20260808 23:44:41.723027 7649 consensus_queue.cc:237] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } }
I20260808 23:44:41.727998 7130 catalog_manager.cc:5495] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 4453151643db4593b58971413bed31ee (127.18.186.131). New cstate: current_term: 1 leader_uuid: "4453151643db4593b58971413bed31ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4453151643db4593b58971413bed31ee" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 40371 } health_report { overall_health: HEALTHY } } }
I20260808 23:44:41.769558 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.764057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57920 (local address 127.18.186.131:40371)
0808 23:44:41.764284 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.764290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.764311 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:44:41.764492 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.764496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.764573 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.764730 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:44:41.765133 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.765839 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.767246 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.767696 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.767836 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.768462 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:44:41.768942 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:44:41.769140 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.769294 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.769339 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:44:41.778054 7434 raft_consensus.cc:1261] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 1 FOLLOWER]: Refusing update from remote peer 4453151643db4593b58971413bed31ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:41.778053 7288 raft_consensus.cc:1261] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 1 FOLLOWER]: Refusing update from remote peer 4453151643db4593b58971413bed31ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:41.779096 7653 consensus_queue.cc:1035] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "9438d7bf75cd404a9c530e2d8cd6cd3a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 36465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:41.779495 7649 consensus_queue.cc:1035] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "57fe64c21a494d5b84a8d0382bde526f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 45933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 23:44:41.780930 7626 tablet.cc:2369] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:41.791616 7657 mvcc.cc:205] Tried to move back new op lower bound from 7316409064551559168 to 7316409064341639168. Current Snapshot: MvccSnapshot[applied={T|T < 7316409064551559168}]
I20260808 23:44:41.804641 7129 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:41.804919 7129 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:41.808560 7129 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 887d84bfee5c4912bc74411e0ca0c259: Sending DeleteTablet for 3 replicas of tablet b2d0b09962b5432a815439e10f966c17
I20260808 23:44:41.809638 7412 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d0b09962b5432a815439e10f966c17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:41 UTC) from {username='slave'} at 127.0.0.1:39296
I20260808 23:44:41.809751 7553 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d0b09962b5432a815439e10f966c17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:41 UTC) from {username='slave'} at 127.0.0.1:57906
I20260808 23:44:41.810446 7266 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d0b09962b5432a815439e10f966c17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:41 UTC) from {username='slave'} at 127.0.0.1:60844
I20260808 23:44:41.810833 7668 tablet_replica.cc:330] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: stopping tablet replica
I20260808 23:44:41.811254 7668 raft_consensus.cc:2229] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:44:41.811939 7668 raft_consensus.cc:2258] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:41.812670 7667 tablet_replica.cc:330] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: stopping tablet replica
I20260808 23:44:41.812796 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 7197
I20260808 23:44:41.813112 7667 raft_consensus.cc:2229] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:41.813582 7667 raft_consensus.cc:2258] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:41.814502 7668 ts_tablet_manager.cc:1905] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:41.815644 7667 ts_tablet_manager.cc:1905] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:41.823001 7667 ts_tablet_manager.cc:1918] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:41.823212 7667 log.cc:1196] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/b2d0b09962b5432a815439e10f966c17
I20260808 23:44:41.823678 7668 ts_tablet_manager.cc:1918] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:41.823879 7667 ts_tablet_manager.cc:1939] T b2d0b09962b5432a815439e10f966c17 P 9438d7bf75cd404a9c530e2d8cd6cd3a: Deleting consensus metadata
I20260808 23:44:41.823918 7668 log.cc:1196] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/b2d0b09962b5432a815439e10f966c17
I20260808 23:44:41.824589 7668 ts_tablet_manager.cc:1939] T b2d0b09962b5432a815439e10f966c17 P 4453151643db4593b58971413bed31ee: Deleting consensus metadata
I20260808 23:44:41.825999 7114 catalog_manager.cc:4841] TS 9438d7bf75cd404a9c530e2d8cd6cd3a (127.18.186.130:36465): tablet b2d0b09962b5432a815439e10f966c17 (table test-table [id=67d38266fcb24dcd98cdf55f0ce17ce0]) successfully deleted
I20260808 23:44:41.826751 7115 catalog_manager.cc:4841] TS 4453151643db4593b58971413bed31ee (127.18.186.131:40371): tablet b2d0b09962b5432a815439e10f966c17 (table test-table [id=67d38266fcb24dcd98cdf55f0ce17ce0]) successfully deleted
I20260808 23:44:41.828377 7669 tablet_replica.cc:330] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: stopping tablet replica
I20260808 23:44:41.828984 7669 raft_consensus.cc:2229] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:41.829578 7669 raft_consensus.cc:2258] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:41.832171 7669 ts_tablet_manager.cc:1905] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:41.838069 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:41.814282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.18.186.129:45933)
0808 23:44:41.814476 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:41.814483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:41.828248 (+ 13765us) server_negotiation.cc:407] Connection header received
0808 23:44:41.828385 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:41.828395 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:41.828585 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:41.828749 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:41.829738 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.831447 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.833588 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:41.834362 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:41.834475 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:41.836070 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:41.836083 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:41.836103 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:41.836947 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:41.837021 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:41.837054 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:41.837209 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:41.837535 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 23:44:41.837657 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:44:41.839525 7332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:41.990700 7669 ts_tablet_manager.cc:1918] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:41.990986 7669 log.cc:1196] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/b2d0b09962b5432a815439e10f966c17
I20260808 23:44:41.991739 7669 ts_tablet_manager.cc:1939] T b2d0b09962b5432a815439e10f966c17 P 57fe64c21a494d5b84a8d0382bde526f: Deleting consensus metadata
I20260808 23:44:41.991854 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 7197
W20260808 23:44:42.010473 7115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.186.129:45933 (error 108)
I20260808 23:44:42.010727 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 7342
I20260808 23:44:42.012753 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.012249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60868)
0808 23:44:42.012448 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:42.012573 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 23:44:42.013056 7115 catalog_manager.cc:4584] TS 57fe64c21a494d5b84a8d0382bde526f (127.18.186.129:45933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2d0b09962b5432a815439e10f966c17: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
I20260808 23:44:42.018321 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.011618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.18.186.130:36465)
0808 23:44:42.011834 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:42.011843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:42.011950 (+ 107us) server_negotiation.cc:407] Connection header received
0808 23:44:42.012148 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:42.012154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:42.012288 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:42.012404 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:42.012951 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:42.013856 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:42.015138 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:42.015638 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:42.015774 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:42.016970 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:42.016978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:42.016992 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:42.017567 (+ 575us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:42.017613 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:42.017641 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:42.017771 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:42.017910 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 23:44:42.018020 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:44:42.019038 7476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:42.067067 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.066511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60878)
0808 23:44:42.066739 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:42.066870 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 23:44:42.068310 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.067863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60892)
0808 23:44:42.068044 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:42.068145 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 23:44:42.118618 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 7342
I20260808 23:44:42.129017 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.128422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60902)
0808 23:44:42.128679 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:42.128793 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 23:44:42.130229 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.129786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60906)
0808 23:44:42.129969 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:42.130067 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:44:42.138638 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 7485
I20260808 23:44:42.146644 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.139497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57932 (local address 127.18.186.131:40371)
0808 23:44:42.139710 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:42.139716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:42.139739 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:44:42.139919 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:42.139925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:42.140031 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:42.140144 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:42.140545 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:42.141763 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:42.143028 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:42.143738 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:42.143874 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:42.144919 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:42.144932 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:42.144957 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:42.145840 (+ 883us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:42.145919 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:42.145944 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:42.146079 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:42.146137 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 23:44:42.146237 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:44:42.147404 7619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:42.210641 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.210126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60908)
0808 23:44:42.210353 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:42.210470 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 23:44:42.211758 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.211334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60920)
0808 23:44:42.211575 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:42.211645 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260808 23:44:42.270939 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 7485
I20260808 23:44:42.293111 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 7098
I20260808 23:44:42.294261 7163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:42.405730 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:42.405102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:45933 (local address 127.0.0.1:60934)
0808 23:44:42.405311 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:42.405426 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.129:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 23:44:42.406000 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 7098
2026-08-08T23:44:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2851 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:44:42 dist-test-slave-5k54 krb5kdc[7679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:44:42 dist-test-slave-5k54 krb5kdc[7679](info): set up 2 sockets
Aug 08 23:44:42 dist-test-slave-5k54 krb5kdc[7679](info): commencing operation
krb5kdc: starting...
W20260808 23:44:43.587469 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:44:43 dist-test-slave-5k54 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:44:43.888434 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:34515
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:34515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:44.229333 7693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:44.229686 7693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:44.229862 7693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:44.237025 7693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:44.237178 7693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:44.237293 7693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:44.237396 7693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:44.237462 7693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:44.247283 7693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:34515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:34515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:44.248694 7693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:44.250082 7693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:44.262001 7698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:44.262059 7701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:44.262097 7699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:44.263567 7693 server_base.cc:1034] running on GCE node
I20260808 23:44:44.264335 7693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:44.265759 7693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:44.266999 7693 hybrid_clock.cc:648] HybridClock initialized: now 1786232684266950 us; error 58 us; skew 500 ppm
I20260808 23:44:44.267349 7693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:44 dist-test-slave-5k54 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232684, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:44.270551 7693 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:44:44.272146 7693 webserver.cc:460] Webserver started at http://127.18.186.190:39655/ using document root <none> and password file <none>
I20260808 23:44:44.272820 7693 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:44.272979 7693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:44.273288 7693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:44.276366 7693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "4bb8d2d012e94cb69fb0f4228913c2b5"
format_stamp: "Formatted at 2026-08-08 23:44:44 on dist-test-slave-5k54"
server_key: "061a38a22cbea8bdf53df4ad65b0dce5"
server_key_iv: "648f4ab438c4ded5445dec23f9914898"
server_key_version: "encryptionkey@0"
I20260808 23:44:44.277110 7693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "4bb8d2d012e94cb69fb0f4228913c2b5"
format_stamp: "Formatted at 2026-08-08 23:44:44 on dist-test-slave-5k54"
server_key: "061a38a22cbea8bdf53df4ad65b0dce5"
server_key_iv: "648f4ab438c4ded5445dec23f9914898"
server_key_version: "encryptionkey@0"
I20260808 23:44:44.282152 7693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:44:44.285776 7708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:44.287024 7693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 23:44:44.287215 7693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "4bb8d2d012e94cb69fb0f4228913c2b5"
format_stamp: "Formatted at 2026-08-08 23:44:44 on dist-test-slave-5k54"
server_key: "061a38a22cbea8bdf53df4ad65b0dce5"
server_key_iv: "648f4ab438c4ded5445dec23f9914898"
server_key_version: "encryptionkey@0"
I20260808 23:44:44.287427 7693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:44.304189 7693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:44.308068 7693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:44.308367 7693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:44.327212 7693 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:34515
I20260808 23:44:44.327224 7762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:34515 every 8 connection(s)
I20260808 23:44:44.328613 7693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:44.329124 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 7693
I20260808 23:44:44.329342 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:44.330018 19178 external_mini_cluster.cc:1428] Setting key 2c30128806948297df17de874f9af6cf
I20260808 23:44:44.333009 7763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:44:44 dist-test-slave-5k54 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:44.343861 7763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5: Bootstrap starting.
I20260808 23:44:44.347203 7763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:44.348407 7763 log.cc:824] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:44.351708 7763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5: No bootstrap required, opened a new log
I20260808 23:44:44.352602 7766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:44.331961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.18.186.190:34515)
0808 23:44:44.332737 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:44.332749 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:44.332790 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:44:44.334582 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:44.334604 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:44.334975 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:44.335356 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:44.336903 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:44.338245 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:44.339634 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:44.340443 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:44.340662 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:44.345145 (+ 4483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:44.345153 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:44.345165 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:44.345203 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:44.348382 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:44.349259 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:44.349276 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:44.349284 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:44.349546 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:44.349977 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:44.349981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:44.349983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:44.350492 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:44.350714 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:44.351185 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 23:44:44.351588 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":347,"threads_started":1}
I20260808 23:44:44.359586 7763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb8d2d012e94cb69fb0f4228913c2b5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34515 } }
I20260808 23:44:44.359949 7763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:44.360065 7763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb8d2d012e94cb69fb0f4228913c2b5, State: Initialized, Role: FOLLOWER
I20260808 23:44:44.360610 7763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb8d2d012e94cb69fb0f4228913c2b5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34515 } }
I20260808 23:44:44.360846 7763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:44.361003 7763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:44.361169 7763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:44.363481 7763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb8d2d012e94cb69fb0f4228913c2b5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34515 } }
I20260808 23:44:44.363974 7763 leader_election.cc:302] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bb8d2d012e94cb69fb0f4228913c2b5; no voters:
I20260808 23:44:44.364584 7763 leader_election.cc:288] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:44.364702 7769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:44.365439 7769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bb8d2d012e94cb69fb0f4228913c2b5, State: Running, Role: LEADER
I20260808 23:44:44.365978 7769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb8d2d012e94cb69fb0f4228913c2b5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34515 } }
I20260808 23:44:44.366693 7763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:44.369061 7770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bb8d2d012e94cb69fb0f4228913c2b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb8d2d012e94cb69fb0f4228913c2b5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34515 } } }
I20260808 23:44:44.369032 7771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bb8d2d012e94cb69fb0f4228913c2b5. Latest consensus state: current_term: 1 leader_uuid: "4bb8d2d012e94cb69fb0f4228913c2b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb8d2d012e94cb69fb0f4228913c2b5" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 34515 } } }
I20260808 23:44:44.369581 7770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:44.369642 7771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:44.375916 7775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:44.381242 7775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:44.389984 7775 catalog_manager.cc:1347] Generated new cluster ID: a03f4145ecd84191a52f9244887ced56
I20260808 23:44:44.390091 7775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:44.398887 7775 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:44.400774 7775 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:44.408895 7775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5: Generated new TSK 0
I20260808 23:44:44.409865 7775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.186.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
Entry for principal kudu/127.18.186.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab.
I20260808 23:44:44.489097 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34515
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:44.867280 7790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:44.867614 7790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:44.867769 7790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:44.877919 7790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:44.878026 7790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:44.878147 7790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:44:44.888062 7790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.129.keytab
--principal=kudu/127.18.186.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:44.889317 7790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:44.890583 7790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:44.903842 7796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:44.903833 7795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:44.903833 7798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:44.904928 7790 server_base.cc:1034] running on GCE node
I20260808 23:44:44.905522 7790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:44.906487 7790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:44.907680 7790 hybrid_clock.cc:648] HybridClock initialized: now 1786232684907637 us; error 32 us; skew 500 ppm
I20260808 23:44:44.908044 7790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:44 dist-test-slave-5k54 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232684, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:44.911338 7790 init.cc:377] Logged in from keytab as kudu/127.18.186.129@KRBTEST.COM (short username kudu)
I20260808 23:44:44.912846 7790 webserver.cc:460] Webserver started at http://127.18.186.129:39975/ using document root <none> and password file <none>
I20260808 23:44:44.913501 7790 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:44.913627 7790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:44.913938 7790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:44.916862 7790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "99599d0238c74ae5bee59f58f436f16f"
format_stamp: "Formatted at 2026-08-08 23:44:44 on dist-test-slave-5k54"
server_key: "08319422f69c02fa2cccfaa573a0e958"
server_key_iv: "46c0b9e634d0649426b6528aea17ab7d"
server_key_version: "encryptionkey@0"
I20260808 23:44:44.917678 7790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "99599d0238c74ae5bee59f58f436f16f"
format_stamp: "Formatted at 2026-08-08 23:44:44 on dist-test-slave-5k54"
server_key: "08319422f69c02fa2cccfaa573a0e958"
server_key_iv: "46c0b9e634d0649426b6528aea17ab7d"
server_key_version: "encryptionkey@0"
I20260808 23:44:44.922801 7790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 23:44:44.926147 7805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:44.927527 7790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:44:44.927698 7790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "99599d0238c74ae5bee59f58f436f16f"
format_stamp: "Formatted at 2026-08-08 23:44:44 on dist-test-slave-5k54"
server_key: "08319422f69c02fa2cccfaa573a0e958"
server_key_iv: "46c0b9e634d0649426b6528aea17ab7d"
server_key_version: "encryptionkey@0"
I20260808 23:44:44.927883 7790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:44.942773 7790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:44.946497 7790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:44.946724 7790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:44.947630 7790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:44.948922 7790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:44.949007 7790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:44.949097 7790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:44.949141 7790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:44.982933 7790 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:35415
I20260808 23:44:44.983028 7921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:35415 every 8 connection(s)
I20260808 23:44:44.984266 7790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:44:44.989012 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 7790
I20260808 23:44:44.989218 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:44:44.989792 19178 external_mini_cluster.cc:1428] Setting key 221bbe08dcb628d006e6d08f598ac372
Aug 08 23:44:44 dist-test-slave-5k54 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232684, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.129@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:45.006834 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:44.986778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:38947 (local address 127.18.186.190:34515)
0808 23:44:44.987276 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:44.987281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:44.988132 (+ 851us) server_negotiation.cc:407] Connection header received
0808 23:44:44.990725 (+ 2593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:44.990731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:44.990825 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:44.990946 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:44.992873 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:44.993573 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:44.996104 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:44.996496 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:44.996590 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:45.000879 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:45.000884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:45.000888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:45.000921 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:45.003658 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:45.004423 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:45.004428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:45.004430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:45.004560 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:45.005051 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:45.005055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:45.005057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:45.005329 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:45.005496 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:45.006210 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 23:44:45.006443 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":268,"threads_started":1}
I20260808 23:44:45.007896 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:44.987233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34515 (local address 127.18.186.129:38947)
0808 23:44:44.987967 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:44.988005 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:44.990437 (+ 2432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:44.991190 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:44.991198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:44.991844 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:44.992671 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:44.992697 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:44.993747 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:44.993755 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:44.995943 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:44.995962 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:44.996719 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:44.996728 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:44.997042 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:44.997783 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:44.997815 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:45.000622 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:45.003800 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:45.003810 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:45.003827 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:45.004281 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:45.004677 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:45.004682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:45.004686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:45.004908 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:45.005463 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:45.005471 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:45.005827 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 23:44:45.006105 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 23:44:45.006659 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":275,"threads_started":1}
I20260808 23:44:45.009616 7922 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34515
I20260808 23:44:45.009961 7922 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:45.010876 7922 heartbeater.cc:511] Master 127.18.186.190:34515 requested a full tablet report, sending...
I20260808 23:44:45.013522 7726 ts_manager.cc:194] Registered new tserver with Master: 99599d0238c74ae5bee59f58f436f16f (127.18.186.129:35415)
I20260808 23:44:45.015856 7726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.129@KRBTEST.COM'} at 127.18.186.129:38947
WARNING: no policy specified for kudu/127.18.186.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
Entry for principal kudu/127.18.186.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab.
I20260808 23:44:45.062536 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34515
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:45.403538 7929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:45.403806 7929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:45.403900 7929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:45.410904 7929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:45.411008 7929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:45.411201 7929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:44:45.420373 7929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.130.keytab
--principal=kudu/127.18.186.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:45.421566 7929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:45.422890 7929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:45.436506 7934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:45.436503 7938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:45.436520 7936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:45.437620 7929 server_base.cc:1034] running on GCE node
I20260808 23:44:45.438262 7929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:45.439154 7929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:45.440373 7929 hybrid_clock.cc:648] HybridClock initialized: now 1786232685440308 us; error 76 us; skew 500 ppm
I20260808 23:44:45.440696 7929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:45 dist-test-slave-5k54 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232685, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:45.443833 7929 init.cc:377] Logged in from keytab as kudu/127.18.186.130@KRBTEST.COM (short username kudu)
I20260808 23:44:45.445320 7929 webserver.cc:460] Webserver started at http://127.18.186.130:36577/ using document root <none> and password file <none>
I20260808 23:44:45.445998 7929 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:45.446141 7929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:45.446444 7929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:45.449306 7929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "9dd32b4e3bfd4c50bd1c08750824536a"
format_stamp: "Formatted at 2026-08-08 23:44:45 on dist-test-slave-5k54"
server_key: "f4941627ede2601597d394428292fc88"
server_key_iv: "d287ee5a5540c7505c7b34b5ba85c1bb"
server_key_version: "encryptionkey@0"
I20260808 23:44:45.450093 7929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "9dd32b4e3bfd4c50bd1c08750824536a"
format_stamp: "Formatted at 2026-08-08 23:44:45 on dist-test-slave-5k54"
server_key: "f4941627ede2601597d394428292fc88"
server_key_iv: "d287ee5a5540c7505c7b34b5ba85c1bb"
server_key_version: "encryptionkey@0"
I20260808 23:44:45.455085 7929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 23:44:45.458489 7946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:45.459941 7929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:45.460098 7929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "9dd32b4e3bfd4c50bd1c08750824536a"
format_stamp: "Formatted at 2026-08-08 23:44:45 on dist-test-slave-5k54"
server_key: "f4941627ede2601597d394428292fc88"
server_key_iv: "d287ee5a5540c7505c7b34b5ba85c1bb"
server_key_version: "encryptionkey@0"
I20260808 23:44:45.460269 7929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:45.504446 7929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:45.508980 7929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:45.509275 7929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:45.510234 7929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:45.511638 7929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:45.511711 7929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:45.511878 7929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:45.511930 7929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:45.544919 7929 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:40609
I20260808 23:44:45.545008 8062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:40609 every 8 connection(s)
I20260808 23:44:45.546218 7929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:44:45.553279 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 7929
I20260808 23:44:45.553481 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:44:45.553937 19178 external_mini_cluster.cc:1428] Setting key debe3c0dc7c84a3fbdf9be68a8b8d6a2
Aug 08 23:44:45 dist-test-slave-5k54 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232685, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.130@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:45.566736 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:45.548886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:48823 (local address 127.18.186.190:34515)
0808 23:44:45.549365 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:45.549370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:45.550296 (+ 926us) server_negotiation.cc:407] Connection header received
0808 23:44:45.552235 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:45.552241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:45.552337 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:45.552454 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:45.554114 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:45.555139 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:45.556924 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:45.557356 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:45.557496 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:45.560847 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:45.560852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:45.560855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:45.560889 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:45.563687 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:45.564428 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:45.564432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:45.564434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:45.564564 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:45.565026 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:45.565030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:45.565032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:45.565298 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:45.565477 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:45.566128 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 23:44:45.566351 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":215,"threads_started":1}
I20260808 23:44:45.567759 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:45.549446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34515 (local address 127.18.186.130:48823)
0808 23:44:45.550126 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:45.550169 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:45.551957 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:45.552688 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:45.552696 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:45.553228 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:45.553908 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:45.553925 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:45.555290 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:45.555296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:45.556756 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:45.556769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:45.557607 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:45.557615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:45.557840 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:45.558504 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:45.558525 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:45.560618 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:45.563827 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:45.563837 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:45.563853 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:45.564283 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:45.564676 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:45.564681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:45.564685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:45.564896 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:45.565432 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:45.565440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:45.565770 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 23:44:45.566027 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 23:44:45.566528 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":256,"threads_started":1}
I20260808 23:44:45.569245 8063 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34515
I20260808 23:44:45.569569 8063 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:45.570405 8063 heartbeater.cc:511] Master 127.18.186.190:34515 requested a full tablet report, sending...
I20260808 23:44:45.571964 7726 ts_manager.cc:194] Registered new tserver with Master: 9dd32b4e3bfd4c50bd1c08750824536a (127.18.186.130:40609)
I20260808 23:44:45.572762 7726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.130@KRBTEST.COM'} at 127.18.186.130:48823
WARNING: no policy specified for kudu/127.18.186.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
Entry for principal kudu/127.18.186.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab.
I20260808 23:44:45.623405 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34515
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:45.955977 8069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:45.956238 8069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:45.956408 8069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:45.963477 8069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:45.963575 8069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:45.963781 8069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:44:45.973453 8069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.131.keytab
--principal=kudu/127.18.186.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:34515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:45.974687 8069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:45.976122 8069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:45.989037 8075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:45.989099 8074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:45.989394 8077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:45.990191 8069 server_base.cc:1034] running on GCE node
I20260808 23:44:45.990813 8069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:45.991775 8069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:45.992964 8069 hybrid_clock.cc:648] HybridClock initialized: now 1786232685992905 us; error 49 us; skew 500 ppm
I20260808 23:44:45.993276 8069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:45 dist-test-slave-5k54 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232685, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:45.996449 8069 init.cc:377] Logged in from keytab as kudu/127.18.186.131@KRBTEST.COM (short username kudu)
I20260808 23:44:45.997903 8069 webserver.cc:460] Webserver started at http://127.18.186.131:39895/ using document root <none> and password file <none>
I20260808 23:44:45.998567 8069 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:45.998713 8069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:45.999037 8069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:46.001919 8069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "e583e240b4504a488bbce024b5cf12c3"
format_stamp: "Formatted at 2026-08-08 23:44:45 on dist-test-slave-5k54"
server_key: "dee2cb66852f1076bc2a6d1736f55e79"
server_key_iv: "0e8caacdfe068ffa4d0e33063ff868ae"
server_key_version: "encryptionkey@0"
I20260808 23:44:46.002691 8069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "e583e240b4504a488bbce024b5cf12c3"
format_stamp: "Formatted at 2026-08-08 23:44:45 on dist-test-slave-5k54"
server_key: "dee2cb66852f1076bc2a6d1736f55e79"
server_key_iv: "0e8caacdfe068ffa4d0e33063ff868ae"
server_key_version: "encryptionkey@0"
I20260808 23:44:46.007838 8069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 23:44:46.011204 8086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:46.012545 8069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:46.012701 8069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "e583e240b4504a488bbce024b5cf12c3"
format_stamp: "Formatted at 2026-08-08 23:44:45 on dist-test-slave-5k54"
server_key: "dee2cb66852f1076bc2a6d1736f55e79"
server_key_iv: "0e8caacdfe068ffa4d0e33063ff868ae"
server_key_version: "encryptionkey@0"
I20260808 23:44:46.012920 8069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:46.019763 7922 heartbeater.cc:503] Master 127.18.186.190:34515 was elected leader, sending a full tablet report...
I20260808 23:44:46.036185 8069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:46.040035 8069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:46.040323 8069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:46.041286 8069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:46.042639 8069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:46.042745 8069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:46.042901 8069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:46.042976 8069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:46.077085 8069 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:36437
I20260808 23:44:46.077172 8206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:36437 every 8 connection(s)
I20260808 23:44:46.078415 8069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:44:46.082535 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 8069
I20260808 23:44:46.082731 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:44:46.083174 19178 external_mini_cluster.cc:1428] Setting key f4c8e14caf053a5c9600473d1cdf7453
Aug 08 23:44:46 dist-test-slave-5k54 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232685, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.131@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:44:46.098774 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.081040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:40787 (local address 127.18.186.190:34515)
0808 23:44:46.081509 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.081514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.082487 (+ 973us) server_negotiation.cc:407] Connection header received
0808 23:44:46.084694 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.084700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.084802 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.084923 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:46.086659 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.087482 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.088983 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.089408 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.089536 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.093049 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:46.093054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:46.093057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:46.093087 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:46.095821 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.096544 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.096551 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.096555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.096679 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.097104 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.097109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.097113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.097372 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:46.097547 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.098167 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.098381 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":223,"threads_started":1}
I20260808 23:44:46.099761 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.081604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:34515 (local address 127.18.186.131:40787)
0808 23:44:46.082310 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:46.082353 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:46.084375 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:46.085166 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:46.085177 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:46.085946 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:46.086494 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.086511 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.087627 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.087633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.088838 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.088849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.089632 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.089638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.089857 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.090599 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:44:46.090621 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:46.092850 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:46.095964 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:46.095972 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:46.095986 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:46.096426 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:46.096793 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:44:46.096800 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:44:46.096803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:44:46.096988 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:44:46.097504 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:46.097511 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:46.097828 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 23:44:46.098085 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 23:44:46.098586 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":286,"threads_started":1}
I20260808 23:44:46.101166 8207 heartbeater.cc:347] Connected to a master server at 127.18.186.190:34515
I20260808 23:44:46.101476 8207 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:46.102161 8207 heartbeater.cc:511] Master 127.18.186.190:34515 requested a full tablet report, sending...
I20260808 23:44:46.103659 7726 ts_manager.cc:194] Registered new tserver with Master: e583e240b4504a488bbce024b5cf12c3 (127.18.186.131:36437)
I20260808 23:44:46.104414 7726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.186.131@KRBTEST.COM'} at 127.18.186.131:40787
I20260808 23:44:46.110517 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:44:46.127002 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.114620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.18.186.190:34515)
0808 23:44:46.114789 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.114794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.115270 (+ 476us) server_negotiation.cc:407] Connection header received
0808 23:44:46.115414 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.115418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.115535 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.115619 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:46.117171 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.117956 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.119220 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.119670 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.119763 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.122178 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:46.122183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:46.122186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:46.122213 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:46.124705 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.125379 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.125382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.125385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.125506 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.125837 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.125840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.125842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.126139 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:46.126316 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.126617 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.126779 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:44:46.130950 7726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:44:46.137903 7726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:44:46.173175 8222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.149624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:36437 (local address 127.0.0.1:53392)
0808 23:44:46.161386 (+ 11762us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:46.161438 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:46.161657 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:46.165710 (+ 4053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:46.165716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:46.165777 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:46.166326 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.166338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.168509 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.168518 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.170278 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.170291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.172062 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.172068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.172284 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.172343 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 23:44:46.172480 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 23:44:46.172685 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11440,"mutex_wait_us":2,"spinlock_wait_cycles":10112,"thread_start_us":15544,"threads_started":1}
I20260808 23:44:46.173496 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.149945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.18.186.131:36437)
0808 23:44:46.165086 (+ 15141us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.165098 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.165134 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:44:46.165236 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.165246 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.165499 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.166390 (+ 891us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:46.166703 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.168360 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.170436 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.171861 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.172076 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.172274 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.172939 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.173122 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14955,"thread_start_us":14801,"threads_started":1}
I20260808 23:44:46.173815 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.152437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55140 (local address 127.18.186.129:35415)
0808 23:44:46.166187 (+ 13750us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.166196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.166227 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:44:46.166322 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.166329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.166588 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.166788 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:46.167380 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.168929 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.171219 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.172474 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.172668 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.172827 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.173269 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.173452 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13559,"thread_start_us":13373,"threads_started":1}
I20260808 23:44:46.174440 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.151715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:35415 (local address 127.0.0.1:55140)
0808 23:44:46.165424 (+ 13709us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:46.165440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:46.165563 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:46.166777 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:46.166781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:46.166822 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:46.167165 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.167174 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.169099 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.169105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.171073 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.171084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.172669 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.172675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.172848 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.172879 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:44:46.172971 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 23:44:46.173115 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13598,"thread_start_us":11957,"threads_started":1}
I20260808 23:44:46.174504 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.149878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:40609 (local address 127.0.0.1:35590)
0808 23:44:46.164090 (+ 14212us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:46.164113 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:46.164265 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:46.165612 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:46.165623 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:46.165666 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:46.166119 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.166131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.168270 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.168278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.170590 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.170605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.173762 (+ 3157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.173768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.173962 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.174006 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 23:44:46.174097 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 23:44:46.174206 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14006,"thread_start_us":11003,"threads_started":1}
I20260808 23:44:46.175067 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.150327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.18.186.130:40609)
0808 23:44:46.159676 (+ 9349us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.159689 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.164410 (+ 4721us) server_negotiation.cc:407] Connection header received
0808 23:44:46.164534 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.164545 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.164971 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.165263 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:46.166308 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.168114 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.170756 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.173296 (+ 2540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.173451 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.173604 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.174510 (+ 906us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.174702 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9116,"thread_start_us":8941,"threads_started":1}
I20260808 23:44:46.178223 7853 tablet_service.cc:1467] Processing CreateTablet for tablet fb7fb9139d1640b3a3c1dd1a69a69e63 (DEFAULT_TABLE table=test-table [id=b24d6da693c34b48a0925de562aeccf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:46.178242 8136 tablet_service.cc:1467] Processing CreateTablet for tablet fb7fb9139d1640b3a3c1dd1a69a69e63 (DEFAULT_TABLE table=test-table [id=b24d6da693c34b48a0925de562aeccf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:46.178459 7994 tablet_service.cc:1467] Processing CreateTablet for tablet fb7fb9139d1640b3a3c1dd1a69a69e63 (DEFAULT_TABLE table=test-table [id=b24d6da693c34b48a0925de562aeccf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:46.179630 7853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7fb9139d1640b3a3c1dd1a69a69e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:46.179795 7994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7fb9139d1640b3a3c1dd1a69a69e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:46.179790 8136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7fb9139d1640b3a3c1dd1a69a69e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:46.191692 8227 tablet_bootstrap.cc:492] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Bootstrap starting.
I20260808 23:44:46.193689 8228 tablet_bootstrap.cc:492] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Bootstrap starting.
I20260808 23:44:46.195462 8227 tablet_bootstrap.cc:654] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:46.196755 8227 log.cc:824] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:46.197237 8228 tablet_bootstrap.cc:654] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:46.198617 8228 log.cc:824] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:46.199395 8229 tablet_bootstrap.cc:492] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Bootstrap starting.
I20260808 23:44:46.200170 8227 tablet_bootstrap.cc:492] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: No bootstrap required, opened a new log
I20260808 23:44:46.200433 8227 ts_tablet_manager.cc:1397] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 23:44:46.201800 8228 tablet_bootstrap.cc:492] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: No bootstrap required, opened a new log
I20260808 23:44:46.202085 8228 ts_tablet_manager.cc:1397] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 23:44:46.202878 8229 tablet_bootstrap.cc:654] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:46.204191 8229 log.cc:824] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:46.207121 8229 tablet_bootstrap.cc:492] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: No bootstrap required, opened a new log
I20260808 23:44:46.207372 8229 ts_tablet_manager.cc:1397] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 23:44:46.207947 8227 raft_consensus.cc:348] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.208303 8227 raft_consensus.cc:374] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:46.208431 8227 raft_consensus.cc:729] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e583e240b4504a488bbce024b5cf12c3, State: Initialized, Role: FOLLOWER
I20260808 23:44:46.209180 8227 consensus_queue.cc:260] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.212217 8207 heartbeater.cc:503] Master 127.18.186.190:34515 was elected leader, sending a full tablet report...
I20260808 23:44:46.212471 8228 raft_consensus.cc:348] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.212855 8227 ts_tablet_manager.cc:1428] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 23:44:46.212875 8228 raft_consensus.cc:374] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:46.213020 8228 raft_consensus.cc:729] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99599d0238c74ae5bee59f58f436f16f, State: Initialized, Role: FOLLOWER
I20260808 23:44:46.213711 8228 consensus_queue.cc:260] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.215407 8228 ts_tablet_manager.cc:1428] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
I20260808 23:44:46.217960 8229 raft_consensus.cc:348] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.218420 8229 raft_consensus.cc:374] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:46.218534 8229 raft_consensus.cc:729] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dd32b4e3bfd4c50bd1c08750824536a, State: Initialized, Role: FOLLOWER
I20260808 23:44:46.219081 8229 consensus_queue.cc:260] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.220046 8063 heartbeater.cc:503] Master 127.18.186.190:34515 was elected leader, sending a full tablet report...
I20260808 23:44:46.220551 8229 ts_tablet_manager.cc:1428] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
W20260808 23:44:46.236528 7923 tablet.cc:2369] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:46.281796 8235 raft_consensus.cc:482] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:44:46.282173 8235 raft_consensus.cc:504] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.283958 8235 leader_election.cc:288] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e583e240b4504a488bbce024b5cf12c3 (127.18.186.131:36437), 99599d0238c74ae5bee59f58f436f16f (127.18.186.129:35415)
I20260808 23:44:46.293519 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.284470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:35247 (local address 127.18.186.131:36437)
0808 23:44:46.284672 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.284679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.285505 (+ 826us) server_negotiation.cc:407] Connection header received
0808 23:44:46.285697 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.285703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.285780 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.285919 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:46.286708 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.287884 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.290996 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.292320 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.292473 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.292545 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.293124 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.293260 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:44:46.293524 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.284407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:56761 (local address 127.18.186.129:35415)
0808 23:44:46.284619 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.284633 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.285505 (+ 872us) server_negotiation.cc:407] Connection header received
0808 23:44:46.285701 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.285706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.285780 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.285892 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:46.286708 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.287872 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.290996 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.292306 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.292472 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.292545 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.293124 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.293259 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:44:46.293742 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.284359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:36437 (local address 127.18.186.130:35247)
0808 23:44:46.285384 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:46.285402 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:46.285531 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:46.285935 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:46.285939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:46.286011 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:46.286546 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.286555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.288016 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.288019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.290811 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.290827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.292595 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.292601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.292876 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.292960 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 23:44:46.293055 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 23:44:46.293232 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":340,"spinlock_wait_cycles":26496,"thread_start_us":616,"threads_started":1}
I20260808 23:44:46.293799 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.284264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:35415 (local address 127.18.186.130:56761)
0808 23:44:46.285344 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:46.285366 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:46.285569 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:46.285933 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:46.285937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:46.285967 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:46.286511 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.286531 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.288014 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.288019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.290811 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:46.290827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.292587 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.292594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:46.292879 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.292955 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 23:44:46.293055 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 23:44:46.293232 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":256,"spinlock_wait_cycles":24320,"thread_start_us":638,"threads_started":1}
I20260808 23:44:46.295116 8157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7fb9139d1640b3a3c1dd1a69a69e63" candidate_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e583e240b4504a488bbce024b5cf12c3" is_pre_election: true
I20260808 23:44:46.295116 7875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7fb9139d1640b3a3c1dd1a69a69e63" candidate_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99599d0238c74ae5bee59f58f436f16f" is_pre_election: true
I20260808 23:44:46.295753 8157 raft_consensus.cc:2454] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dd32b4e3bfd4c50bd1c08750824536a in term 0.
I20260808 23:44:46.295753 7875 raft_consensus.cc:2454] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dd32b4e3bfd4c50bd1c08750824536a in term 0.
I20260808 23:44:46.296525 7947 leader_election.cc:302] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 99599d0238c74ae5bee59f58f436f16f, 9dd32b4e3bfd4c50bd1c08750824536a, e583e240b4504a488bbce024b5cf12c3; no voters:
I20260808 23:44:46.297019 8235 raft_consensus.cc:2781] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:44:46.297178 8235 raft_consensus.cc:482] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:44:46.297294 8235 raft_consensus.cc:3037] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 0 FOLLOWER]: Advancing to term 1
W20260808 23:44:46.298508 8064 tablet.cc:2369] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:46.299742 8235 raft_consensus.cc:504] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.300333 8235 leader_election.cc:288] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [CANDIDATE]: Term 1 election: Requested vote from peers e583e240b4504a488bbce024b5cf12c3 (127.18.186.131:36437), 99599d0238c74ae5bee59f58f436f16f (127.18.186.129:35415)
I20260808 23:44:46.300809 8157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7fb9139d1640b3a3c1dd1a69a69e63" candidate_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e583e240b4504a488bbce024b5cf12c3"
I20260808 23:44:46.300836 7875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7fb9139d1640b3a3c1dd1a69a69e63" candidate_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99599d0238c74ae5bee59f58f436f16f"
I20260808 23:44:46.301018 8157 raft_consensus.cc:3037] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:46.301034 7875 raft_consensus.cc:3037] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:46.303138 7875 raft_consensus.cc:2454] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dd32b4e3bfd4c50bd1c08750824536a in term 1.
I20260808 23:44:46.303170 8157 raft_consensus.cc:2454] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dd32b4e3bfd4c50bd1c08750824536a in term 1.
I20260808 23:44:46.303573 7949 leader_election.cc:302] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99599d0238c74ae5bee59f58f436f16f, 9dd32b4e3bfd4c50bd1c08750824536a; no voters:
I20260808 23:44:46.303872 8235 raft_consensus.cc:2781] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:46.304268 8235 raft_consensus.cc:686] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 1 LEADER]: Becoming Leader. State: Replica: 9dd32b4e3bfd4c50bd1c08750824536a, State: Running, Role: LEADER
I20260808 23:44:46.304719 8235 consensus_queue.cc:237] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } }
I20260808 23:44:46.309217 7726 catalog_manager.cc:5495] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9dd32b4e3bfd4c50bd1c08750824536a (127.18.186.130). New cstate: current_term: 1 leader_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9dd32b4e3bfd4c50bd1c08750824536a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 40609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 } health_report { overall_health: UNKNOWN } } }
W20260808 23:44:46.330130 8208 tablet.cc:2369] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:46.360445 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.355349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35594 (local address 127.18.186.130:40609)
0808 23:44:46.355622 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.355627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.355656 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:44:46.355836 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.355840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.355930 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.356074 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:44:46.356427 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.357160 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.358468 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.358832 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.358920 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.359315 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:44:46.359894 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:44:46.360012 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.360070 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.360133 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 23:44:46.368588 8157 raft_consensus.cc:1261] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 1 FOLLOWER]: Refusing update from remote peer 9dd32b4e3bfd4c50bd1c08750824536a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:46.368587 7875 raft_consensus.cc:1261] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 1 FOLLOWER]: Refusing update from remote peer 9dd32b4e3bfd4c50bd1c08750824536a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:46.369588 8239 consensus_queue.cc:1035] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e583e240b4504a488bbce024b5cf12c3" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:46.369935 8235 consensus_queue.cc:1035] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99599d0238c74ae5bee59f58f436f16f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 35415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:46.382761 8244 mvcc.cc:205] Tried to move back new op lower bound from 7316409083355250688 to 7316409083107622912. Current Snapshot: MvccSnapshot[applied={T|T < 7316409083355250688}]
I20260808 23:44:46.393836 7726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:46.394059 7726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:46.397730 7726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bb8d2d012e94cb69fb0f4228913c2b5: Sending DeleteTablet for 3 replicas of tablet fb7fb9139d1640b3a3c1dd1a69a69e63
I20260808 23:44:46.398972 8136 tablet_service.cc:1514] Processing DeleteTablet for tablet fb7fb9139d1640b3a3c1dd1a69a69e63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:53392
I20260808 23:44:46.398974 7994 tablet_service.cc:1514] Processing DeleteTablet for tablet fb7fb9139d1640b3a3c1dd1a69a69e63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:35590
I20260808 23:44:46.399048 7853 tablet_service.cc:1514] Processing DeleteTablet for tablet fb7fb9139d1640b3a3c1dd1a69a69e63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:46 UTC) from {username='kudu', principal='kudu/127.18.186.190@KRBTEST.COM'} at 127.0.0.1:55140
I20260808 23:44:46.401889 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 7790
I20260808 23:44:46.411319 8254 tablet_replica.cc:330] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: stopping tablet replica
I20260808 23:44:46.411314 8253 tablet_replica.cc:330] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: stopping tablet replica
I20260808 23:44:46.411904 8254 raft_consensus.cc:2229] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:46.412485 8254 raft_consensus.cc:2258] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:46.412456 8253 raft_consensus.cc:2229] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:46.412920 8253 raft_consensus.cc:2258] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:46.415700 8252 tablet_replica.cc:330] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: stopping tablet replica
I20260808 23:44:46.415714 8253 ts_tablet_manager.cc:1905] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:46.416103 8252 raft_consensus.cc:2229] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:44:46.416944 8252 raft_consensus.cc:2258] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:46.419019 8252 ts_tablet_manager.cc:1905] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:46.424358 8254 ts_tablet_manager.cc:1905] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:46.426800 8252 ts_tablet_manager.cc:1918] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:46.427018 8252 log.cc:1196] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/fb7fb9139d1640b3a3c1dd1a69a69e63
I20260808 23:44:46.427625 8252 ts_tablet_manager.cc:1939] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 9dd32b4e3bfd4c50bd1c08750824536a: Deleting consensus metadata
Aug 08 23:44:46 dist-test-slave-5k54 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.129@KRBTEST.COM
I20260808 23:44:46.429581 8253 ts_tablet_manager.cc:1918] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:46.429574 7712 catalog_manager.cc:4841] TS 9dd32b4e3bfd4c50bd1c08750824536a (127.18.186.130:40609): tablet fb7fb9139d1640b3a3c1dd1a69a69e63 (table test-table [id=b24d6da693c34b48a0925de562aeccf4]) successfully deleted
I20260808 23:44:46.429780 8253 log.cc:1196] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/fb7fb9139d1640b3a3c1dd1a69a69e63
I20260808 23:44:46.430321 8253 ts_tablet_manager.cc:1939] T fb7fb9139d1640b3a3c1dd1a69a69e63 P e583e240b4504a488bbce024b5cf12c3: Deleting consensus metadata
I20260808 23:44:46.431335 8254 ts_tablet_manager.cc:1918] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:46.431561 8254 log.cc:1196] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/fb7fb9139d1640b3a3c1dd1a69a69e63
I20260808 23:44:46.432662 8254 ts_tablet_manager.cc:1939] T fb7fb9139d1640b3a3c1dd1a69a69e63 P 99599d0238c74ae5bee59f58f436f16f: Deleting consensus metadata
I20260808 23:44:46.432843 7709 catalog_manager.cc:4841] TS e583e240b4504a488bbce024b5cf12c3 (127.18.186.131:36437): tablet fb7fb9139d1640b3a3c1dd1a69a69e63 (table test-table [id=b24d6da693c34b48a0925de562aeccf4]) successfully deleted
I20260808 23:44:46.434192 7711 catalog_manager.cc:4841] TS 99599d0238c74ae5bee59f58f436f16f (127.18.186.129:35415): tablet fb7fb9139d1640b3a3c1dd1a69a69e63 (table test-table [id=b24d6da693c34b48a0925de562aeccf4]) successfully deleted
I20260808 23:44:46.437522 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.403311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55156 (local address 127.18.186.129:35415)
0808 23:44:46.403512 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.403518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.417184 (+ 13666us) server_negotiation.cc:407] Connection header received
0808 23:44:46.417697 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.417713 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.417806 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.417881 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:46.419755 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.420863 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.422779 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.423493 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.423594 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.430300 (+ 6706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:46.430318 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:46.430338 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:46.430382 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:46.434571 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.435217 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.435226 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.435229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.435396 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.435794 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.435799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.435803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.436244 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:46.436549 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.436747 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.437143 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:44:46.438172 7917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:46.533005 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 7790
I20260808 23:44:46.556272 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 7929
Aug 08 23:44:46 dist-test-slave-5k54 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.130@KRBTEST.COM
I20260808 23:44:46.578200 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.557513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.18.186.130:40609)
0808 23:44:46.557685 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.557691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.557709 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:44:46.557768 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.557773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.557851 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.557960 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:46.559977 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.561281 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.563189 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.563889 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.564010 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.569963 (+ 5953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:46.569979 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:46.569999 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:46.570054 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:46.574589 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.575631 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.575641 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.575645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.575835 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.576361 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.576366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.576370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.576895 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:46.577272 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.577563 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.577802 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 23:44:46.579315 8058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:46.696811 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 7929
I20260808 23:44:46.719074 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 8069
Aug 08 23:44:46 dist-test-slave-5k54 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.131@KRBTEST.COM
I20260808 23:44:46.734123 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:46.719880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.18.186.131:36437)
0808 23:44:46.720050 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:46.720054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:46.720102 (+ 48us) server_negotiation.cc:407] Connection header received
0808 23:44:46.720267 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:46.720271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:46.720335 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:46.720429 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:46.721717 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.722457 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.723792 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:46.724194 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:46.724287 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:46.728741 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:46.728755 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:46.728771 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:46.728812 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:46.731653 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.732310 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.732317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.732320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.732455 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:46.732823 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:46.732826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:46.732828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:46.733144 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:46.733351 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:46.733587 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 23:44:46.733744 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:44:46.734766 8202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:46.824667 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 8069
I20260808 23:44:46.843647 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 7693
I20260808 23:44:46.844934 7758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:46.959803 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 7693
2026-08-08T23:44:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4556 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T23:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:47Z Disabled control of system clock
I20260808 23:44:47.052430 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:38877
--webserver_interface=127.18.186.190
--webserver_port=0
--builtin_ntp_servers=127.18.186.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:38877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:44:47.388681 8263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:47.388965 8263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:47.389047 8263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:47.396270 8263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:47.396373 8263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:47.396413 8263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:47.396477 8263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:47.396530 8263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:47.405411 8263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:38877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:38877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:47.406644 8263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:47.408501 8263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:47.420881 8269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:47.420929 8270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:47.420830 8272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:47.421835 8263 server_base.cc:1034] running on GCE node
I20260808 23:44:47.422657 8263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:47.424139 8263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:47.425374 8263 hybrid_clock.cc:648] HybridClock initialized: now 1786232687425335 us; error 55 us; skew 500 ppm
I20260808 23:44:47.425725 8263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:47.428007 8263 webserver.cc:460] Webserver started at http://127.18.186.190:38141/ using document root <none> and password file <none>
I20260808 23:44:47.428758 8263 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:47.428897 8263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:47.429220 8263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:47.432212 8263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25"
format_stamp: "Formatted at 2026-08-08 23:44:47 on dist-test-slave-5k54"
server_key: "adaebb66dd17947196e528b24328ceca"
server_key_iv: "b12497b64f4b5e6d647e3d88f2bd09c9"
server_key_version: "encryptionkey@0"
I20260808 23:44:47.433061 8263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25"
format_stamp: "Formatted at 2026-08-08 23:44:47 on dist-test-slave-5k54"
server_key: "adaebb66dd17947196e528b24328ceca"
server_key_iv: "b12497b64f4b5e6d647e3d88f2bd09c9"
server_key_version: "encryptionkey@0"
I20260808 23:44:47.438079 8263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 23:44:47.441560 8278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:47.442869 8263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:44:47.443064 8263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25"
format_stamp: "Formatted at 2026-08-08 23:44:47 on dist-test-slave-5k54"
server_key: "adaebb66dd17947196e528b24328ceca"
server_key_iv: "b12497b64f4b5e6d647e3d88f2bd09c9"
server_key_version: "encryptionkey@0"
I20260808 23:44:47.443281 8263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:47.464900 8263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:47.465749 8263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:47.466058 8263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:47.484990 8263 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:38877
I20260808 23:44:47.485013 8333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:38877 every 8 connection(s)
I20260808 23:44:47.486368 8263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:47.488767 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 8263
I20260808 23:44:47.488965 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:47.489563 19178 external_mini_cluster.cc:1428] Setting key 8784914cf73dbe5bbccf02986902e4e0
I20260808 23:44:47.490873 8334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:47.502557 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:47.491593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45572 (local address 127.18.186.190:38877)
0808 23:44:47.492553 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:47.492563 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:47.492604 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:44:47.494163 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:47.494182 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:47.494626 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:47.495035 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:47.495621 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:47.496956 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:47.498285 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:47.498942 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:47.499145 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:47.500360 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:47.500367 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:47.500389 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:47.500936 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:47.501012 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:47.501034 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:47.501176 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:47.501412 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 23:44:47.501690 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":354,"threads_started":1}
I20260808 23:44:47.503196 8334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25: Bootstrap starting.
I20260808 23:44:47.506666 8334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:47.507731 8334 log.cc:824] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:47.510610 8334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25: No bootstrap required, opened a new log
I20260808 23:44:47.518087 8334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38877 } }
I20260808 23:44:47.518430 8334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:47.518494 8334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d540f8b5cc6431c8b7d8b3b53b83c25, State: Initialized, Role: FOLLOWER
I20260808 23:44:47.518998 8334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38877 } }
I20260808 23:44:47.519197 8334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:47.519296 8334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:47.519415 8334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:47.522060 8334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38877 } }
I20260808 23:44:47.522516 8334 leader_election.cc:302] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d540f8b5cc6431c8b7d8b3b53b83c25; no voters:
I20260808 23:44:47.523053 8334 leader_election.cc:288] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:47.523233 8339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:47.523975 8339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [term 1 LEADER]: Becoming Leader. State: Replica: 7d540f8b5cc6431c8b7d8b3b53b83c25, State: Running, Role: LEADER
I20260808 23:44:47.524567 8339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38877 } }
I20260808 23:44:47.525233 8334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:47.527662 8341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d540f8b5cc6431c8b7d8b3b53b83c25. Latest consensus state: current_term: 1 leader_uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38877 } } }
I20260808 23:44:47.527669 8340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d540f8b5cc6431c8b7d8b3b53b83c25" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 38877 } } }
I20260808 23:44:47.528235 8341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:47.528266 8340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:47.532344 8345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:47.538134 8345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:47.546901 8345 catalog_manager.cc:1347] Generated new cluster ID: 6fb1cc47f4084688832d14dfd363fb6d
I20260808 23:44:47.547042 8345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:47.565402 8345 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:47.567189 8345 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:47.574664 8345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25: Generated new TSK 0
I20260808 23:44:47.575676 8345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:44:47.587183 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.129:0
--local_ip_for_outbound_sockets=127.18.186.129
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38877
--builtin_ntp_servers=127.18.186.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:44:47.923298 8358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:47.923574 8358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:47.923662 8358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:47.930935 8358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:47.931026 8358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:47.931129 8358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.129
I20260808 23:44:47.939980 8358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.186.129
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.129
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:47.941110 8358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:47.942735 8358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:47.955813 8364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:47.955777 8367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:47.956013 8363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:47.956923 8358 server_base.cc:1034] running on GCE node
I20260808 23:44:47.957505 8358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:47.958405 8358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:47.959645 8358 hybrid_clock.cc:648] HybridClock initialized: now 1786232687959595 us; error 59 us; skew 500 ppm
I20260808 23:44:47.959950 8358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:47.967347 8358 webserver.cc:460] Webserver started at http://127.18.186.129:43631/ using document root <none> and password file <none>
I20260808 23:44:47.968052 8358 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:47.968185 8358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:47.968499 8358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:47.971268 8358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/data/instance:
uuid: "b9889b09f22a4ad9838cc6641f20be6f"
format_stamp: "Formatted at 2026-08-08 23:44:47 on dist-test-slave-5k54"
server_key: "b8e6899d145eb5ad3fde14b9c25e5de3"
server_key_iv: "7962a98629b864809b27efda5425ca70"
server_key_version: "encryptionkey@0"
I20260808 23:44:47.972131 8358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance:
uuid: "b9889b09f22a4ad9838cc6641f20be6f"
format_stamp: "Formatted at 2026-08-08 23:44:47 on dist-test-slave-5k54"
server_key: "b8e6899d145eb5ad3fde14b9c25e5de3"
server_key_iv: "7962a98629b864809b27efda5425ca70"
server_key_version: "encryptionkey@0"
I20260808 23:44:47.977130 8358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:44:47.980516 8373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:47.981951 8358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:47.982143 8358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/wal
uuid: "b9889b09f22a4ad9838cc6641f20be6f"
format_stamp: "Formatted at 2026-08-08 23:44:47 on dist-test-slave-5k54"
server_key: "b8e6899d145eb5ad3fde14b9c25e5de3"
server_key_iv: "7962a98629b864809b27efda5425ca70"
server_key_version: "encryptionkey@0"
I20260808 23:44:47.982359 8358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:48.001936 8358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:48.002759 8358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:48.003158 8358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:48.004243 8358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:48.005749 8358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:48.005873 8358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:48.006008 8358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:48.006099 8358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:48.045102 8358 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.129:39685
I20260808 23:44:48.045182 8496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.129:39685 every 8 connection(s)
I20260808 23:44:48.046911 8358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/data/info.pb
I20260808 23:44:48.047209 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 8358
I20260808 23:44:48.047459 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/wal/instance
I20260808 23:44:48.048126 19178 external_mini_cluster.cc:1428] Setting key 92cca3b73e749f8715f43e93e87477c9
I20260808 23:44:48.053622 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.130:0
--local_ip_for_outbound_sockets=127.18.186.130
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38877
--builtin_ntp_servers=127.18.186.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:44:48.063687 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:48.050370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.129:42491 (local address 127.18.186.190:38877)
0808 23:44:48.050957 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:48.050965 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:48.052584 (+ 1619us) server_negotiation.cc:407] Connection header received
0808 23:44:48.055473 (+ 2889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:48.055479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:48.055564 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:48.055670 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:48.057034 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.058173 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.060274 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.060769 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.060904 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:48.062632 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:48.062640 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:48.062648 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:48.062675 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:48.062744 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:48.062766 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:48.062871 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:48.063106 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 23:44:48.063222 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":281,"threads_started":1}
I20260808 23:44:48.064930 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:48.051172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:38877 (local address 127.18.186.129:42491)
0808 23:44:48.052409 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:48.052474 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:48.055140 (+ 2666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:48.055910 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:48.055918 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:48.056009 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:48.056865 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:48.056894 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.058318 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.058323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:48.060135 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:48.060148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.061024 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.061033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:48.061379 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:48.062245 (+ 866us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:48.062251 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:48.062304 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:48.062422 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:48.062883 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:48.062897 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:48.062907 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 23:44:48.063088 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 23:44:48.063540 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":368,"threads_started":1}
I20260808 23:44:48.066622 8497 heartbeater.cc:347] Connected to a master server at 127.18.186.190:38877
I20260808 23:44:48.067013 8497 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:48.068141 8497 heartbeater.cc:511] Master 127.18.186.190:38877 requested a full tablet report, sending...
I20260808 23:44:48.070917 8299 ts_manager.cc:194] Registered new tserver with Master: b9889b09f22a4ad9838cc6641f20be6f (127.18.186.129:39685)
I20260808 23:44:48.073233 8299 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.129:42491
W20260808 23:44:48.390470 8501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:48.390740 8501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:48.390827 8501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:48.397859 8501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:48.397959 8501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:48.398061 8501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.130
I20260808 23:44:48.406806 8501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.186.130
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.130
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:48.408073 8501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:48.409850 8501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:48.423000 8507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:48.423111 8509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:48.423069 8506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:48.423911 8501 server_base.cc:1034] running on GCE node
I20260808 23:44:48.424494 8501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:48.425448 8501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:48.426642 8501 hybrid_clock.cc:648] HybridClock initialized: now 1786232688426584 us; error 57 us; skew 500 ppm
I20260808 23:44:48.426963 8501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:48.429322 8501 webserver.cc:460] Webserver started at http://127.18.186.130:33159/ using document root <none> and password file <none>
I20260808 23:44:48.430015 8501 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:48.430133 8501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:48.430459 8501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:48.433449 8501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/data/instance:
uuid: "7821fdc9392746cdae686bf4b931601a"
format_stamp: "Formatted at 2026-08-08 23:44:48 on dist-test-slave-5k54"
server_key: "6f656e016c2c119185793d32969e31b1"
server_key_iv: "8d4d5f268d224d920d5ecf37ef63efd7"
server_key_version: "encryptionkey@0"
I20260808 23:44:48.434275 8501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance:
uuid: "7821fdc9392746cdae686bf4b931601a"
format_stamp: "Formatted at 2026-08-08 23:44:48 on dist-test-slave-5k54"
server_key: "6f656e016c2c119185793d32969e31b1"
server_key_iv: "8d4d5f268d224d920d5ecf37ef63efd7"
server_key_version: "encryptionkey@0"
I20260808 23:44:48.439517 8501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260808 23:44:48.442929 8515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:48.444416 8501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:44:48.444597 8501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/wal
uuid: "7821fdc9392746cdae686bf4b931601a"
format_stamp: "Formatted at 2026-08-08 23:44:48 on dist-test-slave-5k54"
server_key: "6f656e016c2c119185793d32969e31b1"
server_key_iv: "8d4d5f268d224d920d5ecf37ef63efd7"
server_key_version: "encryptionkey@0"
I20260808 23:44:48.444821 8501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:48.464215 8501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:48.465034 8501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:48.465353 8501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:48.466288 8501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:48.467896 8501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:48.468015 8501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:48.468123 8501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:48.468216 8501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:48.502684 8501 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.130:43545
I20260808 23:44:48.502728 8635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.130:43545 every 8 connection(s)
I20260808 23:44:48.504038 8501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/data/info.pb
I20260808 23:44:48.511253 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 8501
I20260808 23:44:48.511435 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/wal/instance
I20260808 23:44:48.511924 19178 external_mini_cluster.cc:1428] Setting key 454f442b46063bbbaf531718bcb41b9b
I20260808 23:44:48.514873 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.186.131:0
--local_ip_for_outbound_sockets=127.18.186.131
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38877
--builtin_ntp_servers=127.18.186.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:44:48.516607 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:48.506644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.130:47385 (local address 127.18.186.190:38877)
0808 23:44:48.506831 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:48.506836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:48.508144 (+ 1308us) server_negotiation.cc:407] Connection header received
0808 23:44:48.509835 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:48.509838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:48.509901 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:48.510000 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:48.511028 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.511905 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.513837 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.514239 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.514335 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:48.515926 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:48.515931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:48.515934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:48.515957 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:48.515989 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:48.516001 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:48.516071 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:48.516318 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 23:44:48.516393 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:44:48.518049 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:48.507108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:38877 (local address 127.18.186.130:47385)
0808 23:44:48.508009 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:48.508043 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:48.509586 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:48.510194 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:48.510202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:48.510281 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:48.510814 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:48.510861 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.512103 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.512108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:48.513701 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:48.513713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.514470 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.514479 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:48.514799 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:48.515557 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:48.515563 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:48.515628 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:48.515735 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:48.516099 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:48.516113 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:48.516124 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 23:44:48.516305 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 23:44:48.516728 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":303,"threads_started":1}
I20260808 23:44:48.519492 8636 heartbeater.cc:347] Connected to a master server at 127.18.186.190:38877
I20260808 23:44:48.519794 8636 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:48.520540 8636 heartbeater.cc:511] Master 127.18.186.190:38877 requested a full tablet report, sending...
I20260808 23:44:48.521915 8299 ts_manager.cc:194] Registered new tserver with Master: 7821fdc9392746cdae686bf4b931601a (127.18.186.130:43545)
I20260808 23:44:48.522743 8299 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.130:47385
W20260808 23:44:48.846727 8639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:48.847083 8639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:48.847289 8639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:48.854640 8639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:48.854770 8639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:48.854975 8639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.186.131
I20260808 23:44:48.863942 8639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.186.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.186.131
--webserver_port=0
--tserver_master_addrs=127.18.186.190:38877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.186.131
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:48.865270 8639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:48.867057 8639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:48.883865 8645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:48.884068 8644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:48.884213 8647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:48.885596 8639 server_base.cc:1034] running on GCE node
I20260808 23:44:48.886286 8639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:48.887274 8639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:48.888505 8639 hybrid_clock.cc:648] HybridClock initialized: now 1786232688888453 us; error 68 us; skew 500 ppm
I20260808 23:44:48.888825 8639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:44:48.891053 8639 webserver.cc:460] Webserver started at http://127.18.186.131:35105/ using document root <none> and password file <none>
I20260808 23:44:48.891813 8639 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:48.891944 8639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:48.892252 8639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:48.895037 8639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/data/instance:
uuid: "a2e4420298c843719f9efa859977c7c5"
format_stamp: "Formatted at 2026-08-08 23:44:48 on dist-test-slave-5k54"
server_key: "76f7289818182e5c995fc3b9381822a9"
server_key_iv: "a07cbc4d98c6fca745e01da3fbab530c"
server_key_version: "encryptionkey@0"
I20260808 23:44:48.895881 8639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance:
uuid: "a2e4420298c843719f9efa859977c7c5"
format_stamp: "Formatted at 2026-08-08 23:44:48 on dist-test-slave-5k54"
server_key: "76f7289818182e5c995fc3b9381822a9"
server_key_iv: "a07cbc4d98c6fca745e01da3fbab530c"
server_key_version: "encryptionkey@0"
I20260808 23:44:48.900786 8639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 23:44:48.904210 8655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:48.905611 8639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:44:48.905810 8639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/wal
uuid: "a2e4420298c843719f9efa859977c7c5"
format_stamp: "Formatted at 2026-08-08 23:44:48 on dist-test-slave-5k54"
server_key: "76f7289818182e5c995fc3b9381822a9"
server_key_iv: "a07cbc4d98c6fca745e01da3fbab530c"
server_key_version: "encryptionkey@0"
I20260808 23:44:48.906032 8639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:48.924814 8639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:48.925633 8639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:48.925951 8639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:48.926925 8639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:44:48.928386 8639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:44:48.928519 8639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:48.928649 8639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:44:48.928743 8639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:48.962895 8639 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.131:35455
I20260808 23:44:48.962931 8775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.131:35455 every 8 connection(s)
I20260808 23:44:48.964265 8639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/data/info.pb
I20260808 23:44:48.965751 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 8639
I20260808 23:44:48.965951 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/wal/instance
I20260808 23:44:48.966444 19178 external_mini_cluster.cc:1428] Setting key 5cdd02b232320476b375e99312320883
I20260808 23:44:48.977012 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:48.967046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:56995 (local address 127.18.186.190:38877)
0808 23:44:48.967229 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:48.967234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:48.969135 (+ 1901us) server_negotiation.cc:407] Connection header received
0808 23:44:48.971019 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:48.971022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:48.971085 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:48.971164 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:48.972215 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.972964 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.974506 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.974911 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.975016 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:48.976301 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:48.976306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:48.976309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:48.976325 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:48.976355 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:48.976366 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:48.976436 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:48.976697 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 23:44:48.976772 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:44:48.978078 8778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:48.967716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:38877 (local address 127.18.186.131:56995)
0808 23:44:48.968951 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:48.969010 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:48.970769 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:48.971364 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:48.971373 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:48.971514 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:44:48.972077 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:48.972093 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.973122 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.973127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:48.974382 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:48.974394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:48.975106 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:48.975111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:48.975318 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:48.976017 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:44:48.976019 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:44:48.976057 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:44:48.976131 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:44:48.976486 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:44:48.976495 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:44:48.976503 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:44:48.976660 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 23:44:48.976973 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":401,"threads_started":1}
I20260808 23:44:48.979493 8776 heartbeater.cc:347] Connected to a master server at 127.18.186.190:38877
I20260808 23:44:48.979892 8776 heartbeater.cc:464] Registering TS with master...
I20260808 23:44:48.980844 8776 heartbeater.cc:511] Master 127.18.186.190:38877 requested a full tablet report, sending...
I20260808 23:44:48.982399 8299 ts_manager.cc:194] Registered new tserver with Master: a2e4420298c843719f9efa859977c7c5 (127.18.186.131:35455)
I20260808 23:44:48.983174 8299 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.186.131:56995
I20260808 23:44:48.994163 19178 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:44:49.004328 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:48.998239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.18.186.190:38877)
0808 23:44:48.998456 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:48.998463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:48.998920 (+ 457us) server_negotiation.cc:407] Connection header received
0808 23:44:48.999123 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:48.999128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:48.999198 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:48.999301 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:48.999845 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.000532 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.001891 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.002328 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.002469 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.003617 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:49.003622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:49.003626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:49.003643 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:49.003675 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:49.003687 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:49.003786 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.003938 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.004037 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:44:49.008071 8299 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:44:49.015048 8299 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:44:49.045897 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.034510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.18.186.130:43545)
0808 23:44:49.036893 (+ 2383us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.036908 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.036932 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:44:49.037021 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.037026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.037195 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.037347 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:49.038183 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.039142 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.043075 (+ 3933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.044292 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.044460 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.044698 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.045360 (+ 662us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.045551 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2240,"thread_start_us":2151,"threads_started":1}
I20260808 23:44:49.046664 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.034289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:43545 (local address 127.0.0.1:47038)
0808 23:44:49.035628 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:49.035677 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:49.035931 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:49.037425 (+ 1494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:49.037436 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:49.037480 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:49.037976 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.037993 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.039285 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.039293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.042845 (+ 3552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.042862 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.044847 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.044853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.045056 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.045106 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 23:44:49.046163 (+ 1057us) client_negotiation.cc:240] Negotiation successful
0808 23:44:49.046353 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"thread_start_us":5350,"threads_started":1}
I20260808 23:44:49.049243 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.034521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.131:35455 (local address 127.0.0.1:37850)
0808 23:44:49.039895 (+ 5374us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:49.039910 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:49.040059 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:49.041054 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:49.041083 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:49.041110 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:49.041479 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.041488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.043281 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.043288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.045422 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.045437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.047373 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.047380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.047558 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.047594 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:44:49.047663 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 23:44:49.047811 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5261,"thread_start_us":951,"threads_started":1}
I20260808 23:44:49.050401 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.035253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.18.186.131:35455)
0808 23:44:49.036696 (+ 1443us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.036708 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.040203 (+ 3495us) server_negotiation.cc:407] Connection header received
0808 23:44:49.040316 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.040327 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.040648 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.040831 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:49.041651 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.043102 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.045673 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.047150 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.049558 (+ 2408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.049838 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.049911 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.050062 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1212,"thread_start_us":1024,"threads_started":1}
I20260808 23:44:49.051025 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.040040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39685 (local address 127.0.0.1:35722)
0808 23:44:49.040587 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:49.040606 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:49.040719 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:49.042555 (+ 1836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:49.042560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:49.042584 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:49.043050 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.043063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.045093 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.045098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.047709 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.047720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.050110 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.050116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.050289 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.050320 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:44:49.050408 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:44:49.050695 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":1,"thread_start_us":276,"threads_started":1}
I20260808 23:44:49.051162 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.040330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.18.186.129:39685)
0808 23:44:49.042005 (+ 1675us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.042013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.042041 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:44:49.042147 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.042157 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.042351 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.042545 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:49.043223 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.044926 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.047931 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.049583 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.049785 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.049985 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.050674 (+ 689us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.050833 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1527,"thread_start_us":1438,"threads_started":1}
I20260808 23:44:49.050889 8569 tablet_service.cc:1467] Processing CreateTablet for tablet 47eeef0da1244fd58e0b0b57593937ef (DEFAULT_TABLE table=test-table [id=671fe6b5c187444bad7d72a9303bc0e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:49.054613 8569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47eeef0da1244fd58e0b0b57593937ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:49.055076 8427 tablet_service.cc:1467] Processing CreateTablet for tablet 47eeef0da1244fd58e0b0b57593937ef (DEFAULT_TABLE table=test-table [id=671fe6b5c187444bad7d72a9303bc0e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:49.055383 8707 tablet_service.cc:1467] Processing CreateTablet for tablet 47eeef0da1244fd58e0b0b57593937ef (DEFAULT_TABLE table=test-table [id=671fe6b5c187444bad7d72a9303bc0e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:44:49.056478 8427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47eeef0da1244fd58e0b0b57593937ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:49.056790 8707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47eeef0da1244fd58e0b0b57593937ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:49.068782 8796 tablet_bootstrap.cc:492] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: Bootstrap starting.
I20260808 23:44:49.071259 8799 tablet_bootstrap.cc:492] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Bootstrap starting.
I20260808 23:44:49.072711 8796 tablet_bootstrap.cc:654] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:49.074044 8796 log.cc:824] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:49.074927 8799 tablet_bootstrap.cc:654] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:49.076373 8797 tablet_bootstrap.cc:492] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Bootstrap starting.
I20260808 23:44:49.076807 8799 log.cc:824] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:49.077246 8497 heartbeater.cc:503] Master 127.18.186.190:38877 was elected leader, sending a full tablet report...
I20260808 23:44:49.078266 8796 tablet_bootstrap.cc:492] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: No bootstrap required, opened a new log
I20260808 23:44:49.078594 8796 ts_tablet_manager.cc:1397] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 23:44:49.079833 8797 tablet_bootstrap.cc:654] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:49.081077 8797 log.cc:824] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:49.084864 8797 tablet_bootstrap.cc:492] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: No bootstrap required, opened a new log
I20260808 23:44:49.084861 8799 tablet_bootstrap.cc:492] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: No bootstrap required, opened a new log
I20260808 23:44:49.085130 8797 ts_tablet_manager.cc:1397] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 23:44:49.085134 8799 ts_tablet_manager.cc:1397] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260808 23:44:49.088367 8796 raft_consensus.cc:348] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.088721 8796 raft_consensus.cc:374] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:49.088810 8796 raft_consensus.cc:729] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9889b09f22a4ad9838cc6641f20be6f, State: Initialized, Role: FOLLOWER
I20260808 23:44:49.089370 8796 consensus_queue.cc:260] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.090884 8796 ts_tablet_manager.cc:1428] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260808 23:44:49.095341 8799 raft_consensus.cc:348] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.095777 8799 raft_consensus.cc:374] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:49.095876 8799 raft_consensus.cc:729] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e4420298c843719f9efa859977c7c5, State: Initialized, Role: FOLLOWER
I20260808 23:44:49.096462 8799 consensus_queue.cc:260] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.097549 8776 heartbeater.cc:503] Master 127.18.186.190:38877 was elected leader, sending a full tablet report...
I20260808 23:44:49.097848 8797 raft_consensus.cc:348] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.098278 8799 ts_tablet_manager.cc:1428] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260808 23:44:49.098306 8797 raft_consensus.cc:374] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:49.098416 8797 raft_consensus.cc:729] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7821fdc9392746cdae686bf4b931601a, State: Initialized, Role: FOLLOWER
I20260808 23:44:49.098966 8797 consensus_queue.cc:260] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.100215 8636 heartbeater.cc:503] Master 127.18.186.190:38877 was elected leader, sending a full tablet report...
I20260808 23:44:49.100755 8797 ts_tablet_manager.cc:1428] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260808 23:44:49.183351 8804 raft_consensus.cc:482] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:44:49.183758 8804 raft_consensus.cc:504] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.185544 8804 leader_election.cc:288] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7821fdc9392746cdae686bf4b931601a (127.18.186.130:43545), b9889b09f22a4ad9838cc6641f20be6f (127.18.186.129:39685)
I20260808 23:44:49.192971 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.186006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:45971 (local address 127.18.186.130:43545)
0808 23:44:49.186260 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.186266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.186290 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:44:49.186749 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.186755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.186870 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.186967 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:49.187499 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.188488 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.190968 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.192105 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.192273 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.192346 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.192601 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.192724 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:44:49.193209 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.186304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.129:39685 (local address 127.18.186.131:43361)
0808 23:44:49.186968 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:49.186985 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:49.187112 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:49.187455 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:49.187460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:49.187487 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:49.187803 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.187810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.189012 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.189016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.191019 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.191037 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.192574 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.192579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.192732 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.192760 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 23:44:49.192854 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 23:44:49.192964 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":347,"threads_started":1}
I20260808 23:44:49.193289 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.186496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.186.131:43361 (local address 127.18.186.129:39685)
0808 23:44:49.186813 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.186821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.187086 (+ 265us) server_negotiation.cc:407] Connection header received
0808 23:44:49.187240 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.187244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.187318 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.187388 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:44:49.187964 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.188872 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.191200 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.192416 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.192574 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.192636 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.192897 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.193044 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 23:44:49.193377 8778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.185857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.130:43545 (local address 127.18.186.131:45971)
0808 23:44:49.186093 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 23:44:49.186121 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 23:44:49.186263 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:44:49.187070 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:44:49.187075 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:44:49.187115 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:44:49.187314 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.187323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.188622 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.188626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.190812 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:44:49.190827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.192263 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.192267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:44:49.192415 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.192440 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:44:49.192560 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 23:44:49.192696 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":77,"spinlock_wait_cycles":21760}
I20260808 23:44:49.194389 8590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47eeef0da1244fd58e0b0b57593937ef" candidate_uuid: "a2e4420298c843719f9efa859977c7c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7821fdc9392746cdae686bf4b931601a" is_pre_election: true
I20260808 23:44:49.194389 8449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47eeef0da1244fd58e0b0b57593937ef" candidate_uuid: "a2e4420298c843719f9efa859977c7c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9889b09f22a4ad9838cc6641f20be6f" is_pre_election: true
I20260808 23:44:49.194998 8449 raft_consensus.cc:2454] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2e4420298c843719f9efa859977c7c5 in term 0.
I20260808 23:44:49.194994 8590 raft_consensus.cc:2454] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2e4420298c843719f9efa859977c7c5 in term 0.
I20260808 23:44:49.195747 8659 leader_election.cc:302] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2e4420298c843719f9efa859977c7c5, b9889b09f22a4ad9838cc6641f20be6f; no voters:
I20260808 23:44:49.196108 8804 raft_consensus.cc:2781] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:44:49.196240 8804 raft_consensus.cc:482] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:44:49.196370 8804 raft_consensus.cc:3037] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:49.198565 8804 raft_consensus.cc:504] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.199190 8804 leader_election.cc:288] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [CANDIDATE]: Term 1 election: Requested vote from peers 7821fdc9392746cdae686bf4b931601a (127.18.186.130:43545), b9889b09f22a4ad9838cc6641f20be6f (127.18.186.129:39685)
I20260808 23:44:49.199708 8590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47eeef0da1244fd58e0b0b57593937ef" candidate_uuid: "a2e4420298c843719f9efa859977c7c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7821fdc9392746cdae686bf4b931601a"
I20260808 23:44:49.199769 8449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47eeef0da1244fd58e0b0b57593937ef" candidate_uuid: "a2e4420298c843719f9efa859977c7c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9889b09f22a4ad9838cc6641f20be6f"
I20260808 23:44:49.200011 8449 raft_consensus.cc:3037] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:49.200024 8590 raft_consensus.cc:3037] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:49.202229 8449 raft_consensus.cc:2454] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2e4420298c843719f9efa859977c7c5 in term 1.
I20260808 23:44:49.202257 8590 raft_consensus.cc:2454] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2e4420298c843719f9efa859977c7c5 in term 1.
I20260808 23:44:49.202582 8659 leader_election.cc:302] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2e4420298c843719f9efa859977c7c5, b9889b09f22a4ad9838cc6641f20be6f; no voters:
I20260808 23:44:49.202915 8804 raft_consensus.cc:2781] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:49.203368 8804 raft_consensus.cc:686] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 1 LEADER]: Becoming Leader. State: Replica: a2e4420298c843719f9efa859977c7c5, State: Running, Role: LEADER
I20260808 23:44:49.203835 8804 consensus_queue.cc:237] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } }
I20260808 23:44:49.208681 8299 catalog_manager.cc:5495] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2e4420298c843719f9efa859977c7c5 (127.18.186.131). New cstate: current_term: 1 leader_uuid: "a2e4420298c843719f9efa859977c7c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2e4420298c843719f9efa859977c7c5" member_type: VOTER last_known_addr { host: "127.18.186.131" port: 35455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 } health_report { overall_health: UNKNOWN } } }
W20260808 23:44:49.215886 8777 tablet.cc:2369] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:49.238780 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.233433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.18.186.131:35455)
0808 23:44:49.233613 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.233617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.233760 (+ 143us) server_negotiation.cc:407] Connection header received
0808 23:44:49.233923 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.233927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.234008 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.234168 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:44:49.234571 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.235287 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.236684 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.237103 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.237238 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.237712 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:44:49.238185 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:44:49.238340 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.238444 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.238521 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:44:49.247035 8590 raft_consensus.cc:1261] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 1 FOLLOWER]: Refusing update from remote peer a2e4420298c843719f9efa859977c7c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:49.247035 8449 raft_consensus.cc:1261] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 1 FOLLOWER]: Refusing update from remote peer a2e4420298c843719f9efa859977c7c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:44:49.248104 8807 consensus_queue.cc:1035] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9889b09f22a4ad9838cc6641f20be6f" member_type: VOTER last_known_addr { host: "127.18.186.129" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:44:49.248463 8804 consensus_queue.cc:1035] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7821fdc9392746cdae686bf4b931601a" member_type: VOTER last_known_addr { host: "127.18.186.130" port: 43545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 23:44:49.256080 8637 tablet.cc:2369] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:44:49.263005 8812 mvcc.cc:205] Tried to move back new op lower bound from 7316409095144988672 to 7316409094983118848. Current Snapshot: MvccSnapshot[applied={T|T < 7316409095144988672}]
I20260808 23:44:49.280661 8297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:49.280949 8297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:44:49.284633 8297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d540f8b5cc6431c8b7d8b3b53b83c25: Sending DeleteTablet for 3 replicas of tablet 47eeef0da1244fd58e0b0b57593937ef
I20260808 23:44:49.285799 8707 tablet_service.cc:1514] Processing DeleteTablet for tablet 47eeef0da1244fd58e0b0b57593937ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:49 UTC) from {username='slave'} at 127.0.0.1:37850
I20260808 23:44:49.285835 8569 tablet_service.cc:1514] Processing DeleteTablet for tablet 47eeef0da1244fd58e0b0b57593937ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:49 UTC) from {username='slave'} at 127.0.0.1:47038
I20260808 23:44:49.285975 8427 tablet_service.cc:1514] Processing DeleteTablet for tablet 47eeef0da1244fd58e0b0b57593937ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:44:49 UTC) from {username='slave'} at 127.0.0.1:35722
I20260808 23:44:49.287827 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 8358
I20260808 23:44:49.296921 8821 tablet_replica.cc:330] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: stopping tablet replica
I20260808 23:44:49.297646 8821 raft_consensus.cc:2229] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:44:49.298542 8821 raft_consensus.cc:2258] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:49.298729 8822 tablet_replica.cc:330] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: stopping tablet replica
I20260808 23:44:49.299160 8822 raft_consensus.cc:2229] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:49.299655 8823 tablet_replica.cc:330] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: stopping tablet replica
I20260808 23:44:49.299723 8822 raft_consensus.cc:2258] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:49.300725 8823 raft_consensus.cc:2229] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:44:49.301302 8823 raft_consensus.cc:2258] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:44:49.302049 8822 ts_tablet_manager.cc:1905] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:49.304037 8821 ts_tablet_manager.cc:1905] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:49.303401 8823 ts_tablet_manager.cc:1905] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:44:49.309492 8822 ts_tablet_manager.cc:1918] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:49.309659 8822 log.cc:1196] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-1/wal/wals/47eeef0da1244fd58e0b0b57593937ef
I20260808 23:44:49.310211 8822 ts_tablet_manager.cc:1939] T 47eeef0da1244fd58e0b0b57593937ef P 7821fdc9392746cdae686bf4b931601a: Deleting consensus metadata
I20260808 23:44:49.312213 8282 catalog_manager.cc:4841] TS 7821fdc9392746cdae686bf4b931601a (127.18.186.130:43545): tablet 47eeef0da1244fd58e0b0b57593937ef (table test-table [id=671fe6b5c187444bad7d72a9303bc0e1]) successfully deleted
I20260808 23:44:49.313585 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.289177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.18.186.129:39685)
0808 23:44:49.289368 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.289375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.304624 (+ 15249us) server_negotiation.cc:407] Connection header received
0808 23:44:49.305166 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.305171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.305272 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.305344 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:49.305981 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.307215 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.309185 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.309922 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.310035 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.311802 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:49.311812 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:49.311846 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:49.312584 (+ 738us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:49.312645 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:49.312672 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:49.312776 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.313039 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.313160 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:44:49.314713 8492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:49.317164 8821 ts_tablet_manager.cc:1918] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:49.317420 8821 log.cc:1196] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-2/wal/wals/47eeef0da1244fd58e0b0b57593937ef
I20260808 23:44:49.317999 8821 ts_tablet_manager.cc:1939] T 47eeef0da1244fd58e0b0b57593937ef P a2e4420298c843719f9efa859977c7c5: Deleting consensus metadata
I20260808 23:44:49.319746 8280 catalog_manager.cc:4841] TS a2e4420298c843719f9efa859977c7c5 (127.18.186.131:35455): tablet 47eeef0da1244fd58e0b0b57593937ef (table test-table [id=671fe6b5c187444bad7d72a9303bc0e1]) successfully deleted
I20260808 23:44:49.408509 8823 ts_tablet_manager.cc:1918] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:44:49.408807 8823 log.cc:1196] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: Deleting WAL directory at /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786232366675349-19178-0/minicluster-data/ts-0/wal/wals/47eeef0da1244fd58e0b0b57593937ef
I20260808 23:44:49.409433 8823 ts_tablet_manager.cc:1939] T 47eeef0da1244fd58e0b0b57593937ef P b9889b09f22a4ad9838cc6641f20be6f: Deleting consensus metadata
I20260808 23:44:49.411478 8282 catalog_manager.cc:4841] TS b9889b09f22a4ad9838cc6641f20be6f (127.18.186.129:39685): tablet 47eeef0da1244fd58e0b0b57593937ef (table test-table [id=671fe6b5c187444bad7d72a9303bc0e1]) successfully deleted
I20260808 23:44:49.411576 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 8358
I20260808 23:44:49.433198 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 8501
I20260808 23:44:49.440706 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.434082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47048 (local address 127.18.186.130:43545)
0808 23:44:49.434260 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.434268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.434288 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:44:49.434613 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.434628 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.434752 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.434895 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:49.435313 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.436274 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.437547 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.437962 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.438107 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.439268 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:49.439277 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:49.439292 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:49.439939 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:49.440009 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:49.440028 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:49.440149 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.440287 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.440392 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:44:49.441404 8631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:49.545081 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 8501
I20260808 23:44:49.565774 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 8639
I20260808 23:44:49.574302 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:49.566926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37874 (local address 127.18.186.131:35455)
0808 23:44:49.567142 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:49.567148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:49.567171 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:44:49.567247 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:49.567253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:49.567358 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:49.567520 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:49.567935 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.569190 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.570438 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:49.571129 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:49.571292 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:49.572575 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:49.572587 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:49.572608 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:49.573470 (+ 862us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:49.573537 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:49.573564 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:49.573681 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:49.573802 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 23:44:49.573912 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 23:44:49.575119 8771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:49.699368 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 8639
I20260808 23:44:49.721092 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 8263
I20260808 23:44:49.722307 8329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:49.832619 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 8263
2026-08-08T23:44:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2873 ms)
[----------] 6 tests from EncryptionPolicyTest (22542 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:44:49 dist-test-slave-5k54 krb5kdc[8833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:44:49 dist-test-slave-5k54 krb5kdc[8833](info): set up 2 sockets
Aug 08 23:44:49 dist-test-slave-5k54 krb5kdc[8833](info): commencing operation
krb5kdc: starting...
W20260808 23:44:51.017724 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:44:51 dist-test-slave-5k54 krb5kdc[8833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:44:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:44:51.311056 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33481
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:51.648512 8847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:51.648792 8847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:51.648885 8847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:51.656108 8847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:51.656209 8847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:51.656256 8847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:51.656325 8847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:51.656371 8847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:51.666213 8847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33481
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:51.667505 8847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:51.668819 8847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:51.681358 8856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:51.681527 8853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:51.681581 8854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:51.682523 8847 server_base.cc:1034] running on GCE node
I20260808 23:44:51.683324 8847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:51.684783 8847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:51.686004 8847 hybrid_clock.cc:648] HybridClock initialized: now 1786232691685939 us; error 51 us; skew 500 ppm
I20260808 23:44:51.686374 8847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:51 dist-test-slave-5k54 krb5kdc[8833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232691, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:51.689636 8847 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:44:51.691170 8847 webserver.cc:460] Webserver started at http://127.0.0.1:42843/ using document root <none> and password file <none>
I20260808 23:44:51.691936 8847 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:51.692106 8847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:51.692430 8847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:51.695386 8847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "da5f6eac353444d59e5445a716664bf6"
format_stamp: "Formatted at 2026-08-08 23:44:51 on dist-test-slave-5k54"
server_key: "5b017578eb09c0130a97f670ee3c3558"
server_key_iv: "203917902730d7f04800ccc52f3ccf29"
server_key_version: "encryptionkey@0"
I20260808 23:44:51.696254 8847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "da5f6eac353444d59e5445a716664bf6"
format_stamp: "Formatted at 2026-08-08 23:44:51 on dist-test-slave-5k54"
server_key: "5b017578eb09c0130a97f670ee3c3558"
server_key_iv: "203917902730d7f04800ccc52f3ccf29"
server_key_version: "encryptionkey@0"
I20260808 23:44:51.701409 8847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:44:51.704936 8863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:51.706264 8847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:44:51.706446 8847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "da5f6eac353444d59e5445a716664bf6"
format_stamp: "Formatted at 2026-08-08 23:44:51 on dist-test-slave-5k54"
server_key: "5b017578eb09c0130a97f670ee3c3558"
server_key_iv: "203917902730d7f04800ccc52f3ccf29"
server_key_version: "encryptionkey@0"
I20260808 23:44:51.706647 8847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:51.721655 8847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:51.725868 8847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:51.726168 8847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:51.745378 8847 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33481
I20260808 23:44:51.745467 8922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33481 every 8 connection(s)
I20260808 23:44:51.746734 8847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:51.748601 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 8847
I20260808 23:44:51.748838 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:51.749529 19178 external_mini_cluster.cc:1428] Setting key 712b5f52c123ea3920bddc5ac4161f72
I20260808 23:44:51.750900 8923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:51.762714 8923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6: Bootstrap starting.
Aug 08 23:44:51 dist-test-slave-5k54 krb5kdc[8833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:44:51.766628 8923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:51.767808 8923 log.cc:824] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:51.771016 8923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6: No bootstrap required, opened a new log
I20260808 23:44:51.772531 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:51.751501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43195 (local address 127.0.0.1:33481)
0808 23:44:51.752258 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:51.752268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:51.752309 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:44:51.753878 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:51.753907 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:51.754293 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:51.754711 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:51.756310 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:51.757782 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:51.759006 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:51.759772 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:51.759990 (+ 218us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:51.764588 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:51.764595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:51.764608 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:51.764649 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:51.768102 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:51.769037 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:51.769047 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:51.769057 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:51.769229 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:51.769751 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:51.769755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:51.769757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:51.770268 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:51.770505 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:51.771189 (+ 684us) server_negotiation.cc:299] Negotiation successful
0808 23:44:51.771595 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":345,"threads_started":1}
I20260808 23:44:51.776731 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:44:51.780419 8923 raft_consensus.cc:348] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5f6eac353444d59e5445a716664bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260808 23:44:51.780952 8923 raft_consensus.cc:374] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:51.781091 8923 raft_consensus.cc:729] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da5f6eac353444d59e5445a716664bf6, State: Initialized, Role: FOLLOWER
I20260808 23:44:51.781826 8923 consensus_queue.cc:260] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5f6eac353444d59e5445a716664bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260808 23:44:51.782132 8923 raft_consensus.cc:388] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:51.782286 8923 raft_consensus.cc:482] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:51.782497 8923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:51.786070 8923 raft_consensus.cc:504] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5f6eac353444d59e5445a716664bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260808 23:44:51.786680 8923 leader_election.cc:302] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da5f6eac353444d59e5445a716664bf6; no voters:
I20260808 23:44:51.787372 8923 leader_election.cc:288] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:51.787513 8933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:51.788192 8933 raft_consensus.cc:686] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [term 1 LEADER]: Becoming Leader. State: Replica: da5f6eac353444d59e5445a716664bf6, State: Running, Role: LEADER
I20260808 23:44:51.788836 8933 consensus_queue.cc:237] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5f6eac353444d59e5445a716664bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260808 23:44:51.790894 8923 sys_catalog.cc:564] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:51.791857 8934 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da5f6eac353444d59e5445a716664bf6. Latest consensus state: current_term: 1 leader_uuid: "da5f6eac353444d59e5445a716664bf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5f6eac353444d59e5445a716664bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } } }
I20260808 23:44:51.791882 8935 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da5f6eac353444d59e5445a716664bf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5f6eac353444d59e5445a716664bf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } } }
I20260808 23:44:51.792335 8934 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:51.792335 8935 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:51.793285 8938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:51.804656 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:51.782877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58721 (local address 127.0.0.1:33481)
0808 23:44:51.783124 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:51.783133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:51.783620 (+ 487us) server_negotiation.cc:407] Connection header received
0808 23:44:51.783831 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:51.783835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:51.783899 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:51.783988 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:51.786078 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:51.786940 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:51.789185 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:51.789856 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:51.789971 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:51.794468 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:51.794478 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:51.794486 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:51.794546 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:51.799292 (+ 4746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:51.800613 (+ 1321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:51.800622 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:51.800628 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:51.800877 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:51.801633 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:51.801641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:51.801647 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:51.802144 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:51.802536 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:51.803505 (+ 969us) server_negotiation.cc:299] Negotiation successful
0808 23:44:51.804117 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 23:44:51.810163 8938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:51.820574 8938 catalog_manager.cc:1347] Generated new cluster ID: 9f3c6022c134422eaacdbabb6ab21466
I20260808 23:44:51.820710 8938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:51.842064 8938 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:51.843864 8938 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:51.850672 8938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da5f6eac353444d59e5445a716664bf6: Generated new TSK 0
I20260808 23:44:51.851732 8938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:44:51.868522 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 8847
I20260808 23:44:51.869982 8917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:51.963833 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 8847
W20260808 23:44:51.985093 8830 connection.cc:535] client connection to 127.0.0.1:33481 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33481): Connection reset by peer (error 104)
2026-08-08T23:44:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2123 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:44:52 dist-test-slave-5k54 krb5kdc[8955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:44:52 dist-test-slave-5k54 krb5kdc[8955](info): set up 2 sockets
Aug 08 23:44:52 dist-test-slave-5k54 krb5kdc[8955](info): commencing operation
krb5kdc: starting...
W20260808 23:44:53.146059 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:44:53 dist-test-slave-5k54 krb5kdc[8955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:44:53.442523 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37585
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:53.780501 8968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:53.780791 8968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:53.781009 8968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:53.788096 8968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:53.788226 8968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:53.788308 8968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:53.788417 8968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:53.788491 8968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:53.797933 8968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:53.799189 8968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:53.800496 8968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:53.812575 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:53.812659 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:53.812654 8976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:53.813361 8968 server_base.cc:1034] running on GCE node
I20260808 23:44:53.814138 8968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:53.815550 8968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:53.816773 8968 hybrid_clock.cc:648] HybridClock initialized: now 1786232693816704 us; error 39 us; skew 500 ppm
I20260808 23:44:53.817253 8968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:53 dist-test-slave-5k54 krb5kdc[8955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232693, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:53.825083 8968 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:44:53.826572 8968 webserver.cc:460] Webserver started at http://127.0.0.1:39461/ using document root <none> and password file <none>
I20260808 23:44:53.827268 8968 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:53.827416 8968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:53.827781 8968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:53.830746 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "9bd26811a4aa44efb180e49f08388349"
format_stamp: "Formatted at 2026-08-08 23:44:53 on dist-test-slave-5k54"
server_key: "67f742d79ab79abe648c1eebd004dda0"
server_key_iv: "66d54dbb90194798d62b391c844ef06c"
server_key_version: "encryptionkey@0"
I20260808 23:44:53.831621 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "9bd26811a4aa44efb180e49f08388349"
format_stamp: "Formatted at 2026-08-08 23:44:53 on dist-test-slave-5k54"
server_key: "67f742d79ab79abe648c1eebd004dda0"
server_key_iv: "66d54dbb90194798d62b391c844ef06c"
server_key_version: "encryptionkey@0"
I20260808 23:44:53.836761 8968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 23:44:53.840365 8984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:53.841676 8968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:44:53.841866 8968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "9bd26811a4aa44efb180e49f08388349"
format_stamp: "Formatted at 2026-08-08 23:44:53 on dist-test-slave-5k54"
server_key: "67f742d79ab79abe648c1eebd004dda0"
server_key_iv: "66d54dbb90194798d62b391c844ef06c"
server_key_version: "encryptionkey@0"
I20260808 23:44:53.842072 8968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:53.861179 8968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:53.865283 8968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:53.865574 8968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:53.883785 8968 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37585
I20260808 23:44:53.883834 9043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37585 every 8 connection(s)
I20260808 23:44:53.885188 8968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:53.888912 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:53.891088 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 8968
I20260808 23:44:53.891299 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:53.891904 19178 external_mini_cluster.cc:1428] Setting key 4ddd68fdb09db0944ea634c1fa2ef78a
I20260808 23:44:53.901437 9044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349: Bootstrap starting.
I20260808 23:44:53.905151 9044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:53.905553 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:53.893722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60845 (local address 127.0.0.1:37585)
0808 23:44:53.894591 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:53.894602 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:53.894643 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:44:53.896392 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:53.896414 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:53.896811 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:53.897132 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:53.897761 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:53.899134 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:53.900959 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:53.901679 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:53.901811 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:53.903124 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:53.903131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:53.903146 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:53.903773 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:53.903847 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:53.903875 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:53.903988 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:53.904292 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 23:44:53.904488 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":393,"threads_started":1}
I20260808 23:44:53.906312 9044 log.cc:824] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:53.909549 9044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349: No bootstrap required, opened a new log
I20260808 23:44:53.909626 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:44:53.918053 9044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd26811a4aa44efb180e49f08388349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37585 } }
I20260808 23:44:53.918473 9044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:53.918591 9044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd26811a4aa44efb180e49f08388349, State: Initialized, Role: FOLLOWER
I20260808 23:44:53.919143 9044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd26811a4aa44efb180e49f08388349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37585 } }
I20260808 23:44:53.919411 9044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:53.919628 9044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:53.919812 9044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:53.922434 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:53.914079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55061 (local address 127.0.0.1:37585)
0808 23:44:53.914272 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:53.914277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:53.915991 (+ 1714us) server_negotiation.cc:407] Connection header received
0808 23:44:53.916142 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:53.916147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:53.916234 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:53.916340 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:53.916785 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:53.917969 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:53.919559 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:53.920293 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:53.920439 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:53.921674 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:53.921679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:53.921682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:53.921699 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:53.921737 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:53.921752 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:53.921839 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:53.922040 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 23:44:53.922082 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:44:53.923074 9044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd26811a4aa44efb180e49f08388349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37585 } }
I20260808 23:44:53.923692 9044 leader_election.cc:302] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bd26811a4aa44efb180e49f08388349; no voters:
I20260808 23:44:53.924280 9044 leader_election.cc:288] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:53.924417 9054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:53.925130 9054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [term 1 LEADER]: Becoming Leader. State: Replica: 9bd26811a4aa44efb180e49f08388349, State: Running, Role: LEADER
I20260808 23:44:53.925693 9054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd26811a4aa44efb180e49f08388349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37585 } }
I20260808 23:44:53.926419 9044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:53.929240 9056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bd26811a4aa44efb180e49f08388349. Latest consensus state: current_term: 1 leader_uuid: "9bd26811a4aa44efb180e49f08388349" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd26811a4aa44efb180e49f08388349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37585 } } }
I20260808 23:44:53.929740 9056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:53.934175 9055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bd26811a4aa44efb180e49f08388349" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd26811a4aa44efb180e49f08388349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37585 } } }
I20260808 23:44:53.934394 9055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:53.940261 9063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:53.944855 9063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:53.953593 9063 catalog_manager.cc:1347] Generated new cluster ID: 8d1bc623ea834c749163c5b5aca16184
I20260808 23:44:53.953709 9063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:53.961817 9063 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:53.963374 9063 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:53.971843 9063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bd26811a4aa44efb180e49f08388349: Generated new TSK 0
I20260808 23:44:53.972872 9063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:44:53.988652 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 8968
W20260808 23:44:53.990151 9038 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60845
W20260808 23:44:53.990908 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 8968: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:44:53.991122 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 8968
W20260808 23:44:54.011816 8953 connection.cc:535] client connection to 127.0.0.1:37585 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37585): Connection reset by peer (error 104)
2026-08-08T23:44:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2026 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:44:54 dist-test-slave-5k54 krb5kdc[9075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:44:54 dist-test-slave-5k54 krb5kdc[9075](info): set up 2 sockets
Aug 08 23:44:54 dist-test-slave-5k54 krb5kdc[9075](info): commencing operation
krb5kdc: starting...
W20260808 23:44:55.170979 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:44:55 dist-test-slave-5k54 krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:44:55.474416 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33869
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:55.820758 9089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:55.821064 9089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:55.821159 9089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:55.828367 9089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:55.828487 9089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:55.828534 9089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:55.828603 9089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:55.828647 9089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:55.839237 9089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33869
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:55.840560 9089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:55.841843 9089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:55.854149 9097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:55.854096 9099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:55.854174 9096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:55.854894 9089 server_base.cc:1034] running on GCE node
I20260808 23:44:55.855770 9089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:55.857154 9089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:55.858407 9089 hybrid_clock.cc:648] HybridClock initialized: now 1786232695858378 us; error 55 us; skew 500 ppm
I20260808 23:44:55.858743 9089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:55 dist-test-slave-5k54 krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232695, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:55.861934 9089 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:44:55.863564 9089 webserver.cc:460] Webserver started at http://127.0.0.1:41143/ using document root <none> and password file <none>
I20260808 23:44:55.864233 9089 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:55.864382 9089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:55.864748 9089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:55.867978 9089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "912cdb7ea0b840168aa43ab984c6a0c2"
format_stamp: "Formatted at 2026-08-08 23:44:55 on dist-test-slave-5k54"
server_key: "5083c65af55b3e128f1649a72a6e8e2e"
server_key_iv: "8813c20a65ade3b34fa110df1583e0aa"
server_key_version: "encryptionkey@0"
I20260808 23:44:55.868773 9089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "912cdb7ea0b840168aa43ab984c6a0c2"
format_stamp: "Formatted at 2026-08-08 23:44:55 on dist-test-slave-5k54"
server_key: "5083c65af55b3e128f1649a72a6e8e2e"
server_key_iv: "8813c20a65ade3b34fa110df1583e0aa"
server_key_version: "encryptionkey@0"
I20260808 23:44:55.874119 9089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:44:55.877635 9106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:55.878899 9089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:44:55.879072 9089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "912cdb7ea0b840168aa43ab984c6a0c2"
format_stamp: "Formatted at 2026-08-08 23:44:55 on dist-test-slave-5k54"
server_key: "5083c65af55b3e128f1649a72a6e8e2e"
server_key_iv: "8813c20a65ade3b34fa110df1583e0aa"
server_key_version: "encryptionkey@0"
I20260808 23:44:55.879264 9089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:55.884977 9089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:55.889297 9089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:55.889596 9089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:55.908715 9089 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33869
I20260808 23:44:55.908757 9163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33869 every 8 connection(s)
I20260808 23:44:55.910076 9089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:55.913786 9164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:44:55.914009 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 9089
I20260808 23:44:55.914204 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:55.914652 19178 external_mini_cluster.cc:1428] Setting key 7aa9ec70df711438a53c638d0044a404
I20260808 23:44:55.922981 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:55.916503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55361 (local address 127.0.0.1:33869)
0808 23:44:55.917414 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:55.917429 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:55.917480 (+ 51us) server_negotiation.cc:407] Connection header received
0808 23:44:55.919052 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:55.919089 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:55.919562 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:55.920019 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:55.921018 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:55.921027 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:55.921044 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:55.921061 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:55.921148 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:55.921176 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:55.921428 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:55.921562 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 23:44:55.921755 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":374,"threads_started":1}
I20260808 23:44:55.926748 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:44:55.928777 9164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2: Bootstrap starting.
I20260808 23:44:55.932327 9164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:55.933755 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:55.931242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60747 (local address 127.0.0.1:33869)
0808 23:44:55.931461 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:55.931467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:55.931775 (+ 308us) server_negotiation.cc:407] Connection header received
0808 23:44:55.931908 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:55.931913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:55.931973 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:55.932082 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:55.933060 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:55.933066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:55.933070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:44:55.933090 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:55.933137 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:44:55.933158 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:55.933280 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:55.933397 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 23:44:55.933460 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 23:44:55.933753 9164 log.cc:824] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:55.936659 9164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2: No bootstrap required, opened a new log
I20260808 23:44:55.943838 9164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cdb7ea0b840168aa43ab984c6a0c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260808 23:44:55.944168 9164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:55.944248 9164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912cdb7ea0b840168aa43ab984c6a0c2, State: Initialized, Role: FOLLOWER
I20260808 23:44:55.944824 9164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cdb7ea0b840168aa43ab984c6a0c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260808 23:44:55.945041 9164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:55.945139 9164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:55.945253 9164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:55.947394 9164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cdb7ea0b840168aa43ab984c6a0c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260808 23:44:55.947892 9164 leader_election.cc:302] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 912cdb7ea0b840168aa43ab984c6a0c2; no voters:
I20260808 23:44:55.948390 9164 leader_election.cc:288] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:55.948526 9175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:55.949226 9175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [term 1 LEADER]: Becoming Leader. State: Replica: 912cdb7ea0b840168aa43ab984c6a0c2, State: Running, Role: LEADER
I20260808 23:44:55.949743 9175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cdb7ea0b840168aa43ab984c6a0c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260808 23:44:55.950448 9164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:55.952829 9177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 912cdb7ea0b840168aa43ab984c6a0c2. Latest consensus state: current_term: 1 leader_uuid: "912cdb7ea0b840168aa43ab984c6a0c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cdb7ea0b840168aa43ab984c6a0c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } } }
I20260808 23:44:55.953337 9177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:55.956264 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "912cdb7ea0b840168aa43ab984c6a0c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cdb7ea0b840168aa43ab984c6a0c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } } }
I20260808 23:44:55.956622 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:55.957391 9185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:55.962603 9185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:55.971295 9185 catalog_manager.cc:1347] Generated new cluster ID: e124713b33294996aff49dcd13735608
I20260808 23:44:55.971405 9185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:55.985757 9185 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:55.986198 9185 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:56.005218 9185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 912cdb7ea0b840168aa43ab984c6a0c2: Generated new TSK 0
I20260808 23:44:56.006263 9185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:44:56.016921 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9089
W20260808 23:44:56.018229 9159 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55361
W20260808 23:44:56.019145 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9089: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:44:56.019302 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 9089
2026-08-08T23:44:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2027 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:44:56 dist-test-slave-5k54 krb5kdc[9198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:44:56 dist-test-slave-5k54 krb5kdc[9198](info): set up 2 sockets
Aug 08 23:44:56 dist-test-slave-5k54 krb5kdc[9198](info): commencing operation
krb5kdc: starting...
W20260808 23:44:57.183492 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.007s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:44:57 dist-test-slave-5k54 krb5kdc[9198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:44:57.482311 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45793
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:57.825873 9212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:57.826169 9212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:57.826326 9212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:57.833761 9212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:57.833870 9212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:57.833950 9212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:57.834043 9212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:57.834101 9212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:57.844622 9212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:57.845971 9212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:57.847332 9212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:44:57.859416 9218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:57.859282 9220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:44:57.859498 9217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:44:57.860617 9212 server_base.cc:1034] running on GCE node
I20260808 23:44:57.861331 9212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:44:57.862614 9212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:44:57.863816 9212 hybrid_clock.cc:648] HybridClock initialized: now 1786232697863757 us; error 59 us; skew 500 ppm
I20260808 23:44:57.864112 9212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:44:57 dist-test-slave-5k54 krb5kdc[9198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232697, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:44:57.867349 9212 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:44:57.868965 9212 webserver.cc:460] Webserver started at http://127.0.0.1:43467/ using document root <none> and password file <none>
I20260808 23:44:57.869643 9212 fs_manager.cc:359] Metadata directory not provided
I20260808 23:44:57.869771 9212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:44:57.870090 9212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:44:57.873191 9212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "730f782e545a4f329627d50383ccbc20"
format_stamp: "Formatted at 2026-08-08 23:44:57 on dist-test-slave-5k54"
server_key: "3968eb5620a72f2bb3a4c2df991632db"
server_key_iv: "02b14822d5ba2171be87a456b61aaf48"
server_key_version: "encryptionkey@0"
I20260808 23:44:57.873975 9212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "730f782e545a4f329627d50383ccbc20"
format_stamp: "Formatted at 2026-08-08 23:44:57 on dist-test-slave-5k54"
server_key: "3968eb5620a72f2bb3a4c2df991632db"
server_key_iv: "02b14822d5ba2171be87a456b61aaf48"
server_key_version: "encryptionkey@0"
I20260808 23:44:57.879069 9212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:44:57.882580 9228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:44:57.884080 9212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:44:57.884299 9212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "730f782e545a4f329627d50383ccbc20"
format_stamp: "Formatted at 2026-08-08 23:44:57 on dist-test-slave-5k54"
server_key: "3968eb5620a72f2bb3a4c2df991632db"
server_key_iv: "02b14822d5ba2171be87a456b61aaf48"
server_key_version: "encryptionkey@0"
I20260808 23:44:57.884516 9212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:44:57.903923 9212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:44:57.908109 9212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:44:57.908417 9212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:44:57.929041 9212 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45793
I20260808 23:44:57.929072 9283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45793 every 8 connection(s)
I20260808 23:44:57.930485 9212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:44:57.932076 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 9212
I20260808 23:44:57.932287 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:44:57.932832 19178 external_mini_cluster.cc:1428] Setting key 1342c17c0a8d0501998ee8f5b33c18f1
I20260808 23:44:57.935171 9284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:44:57 dist-test-slave-5k54 krb5kdc[9198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:44:57.948838 9284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20: Bootstrap starting.
I20260808 23:44:57.953059 9284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20: Neither blocks nor log segments found. Creating new log.
I20260808 23:44:57.954305 9284 log.cc:824] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20: Log is configured to *not* fsync() on all Append() calls
I20260808 23:44:57.957732 9284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20: No bootstrap required, opened a new log
I20260808 23:44:57.958073 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:57.934691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34227 (local address 127.0.0.1:45793)
0808 23:44:57.935603 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:57.935614 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:57.935657 (+ 43us) server_negotiation.cc:407] Connection header received
0808 23:44:57.937411 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:57.937435 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:57.937863 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:57.938216 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:57.940090 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:57.941584 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:57.943378 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:57.944196 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:57.944349 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:57.949530 (+ 5181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:57.949540 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:57.949558 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:57.949604 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:57.953658 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:57.954353 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:57.954361 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:57.954368 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:57.954569 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:57.955051 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:57.955059 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:57.955062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:57.955678 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:57.955969 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:57.956340 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 23:44:57.956773 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":387,"threads_started":1}
I20260808 23:44:57.961968 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:44:57.966527 9284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730f782e545a4f329627d50383ccbc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } }
I20260808 23:44:57.966951 9284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:44:57.967073 9284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730f782e545a4f329627d50383ccbc20, State: Initialized, Role: FOLLOWER
I20260808 23:44:57.967784 9284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730f782e545a4f329627d50383ccbc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } }
I20260808 23:44:57.968102 9284 raft_consensus.cc:388] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:44:57.968246 9284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:44:57.968463 9284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:44:57.971336 9284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730f782e545a4f329627d50383ccbc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } }
I20260808 23:44:57.971920 9284 leader_election.cc:302] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 730f782e545a4f329627d50383ccbc20; no voters:
I20260808 23:44:57.972597 9284 leader_election.cc:288] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:44:57.972699 9295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:44:57.973423 9295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [term 1 LEADER]: Becoming Leader. State: Replica: 730f782e545a4f329627d50383ccbc20, State: Running, Role: LEADER
I20260808 23:44:57.974112 9295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730f782e545a4f329627d50383ccbc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } }
I20260808 23:44:57.977483 9284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:44:57.977376 9297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "730f782e545a4f329627d50383ccbc20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730f782e545a4f329627d50383ccbc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } } }
I20260808 23:44:57.977825 9297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:57.978289 9296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 730f782e545a4f329627d50383ccbc20. Latest consensus state: current_term: 1 leader_uuid: "730f782e545a4f329627d50383ccbc20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730f782e545a4f329627d50383ccbc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45793 } } }
I20260808 23:44:57.978995 9296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20 [sys.catalog]: This master's current role is: LEADER
I20260808 23:44:57.983556 9302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:44:57.984597 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:44:57.966700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43121 (local address 127.0.0.1:45793)
0808 23:44:57.966956 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 23:44:57.966963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:44:57.967518 (+ 555us) server_negotiation.cc:407] Connection header received
0808 23:44:57.967689 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:44:57.967693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:44:57.967748 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:44:57.967838 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:44:57.969807 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:57.970543 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:57.972477 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:44:57.973318 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:44:57.973480 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:44:57.976861 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:44:57.976870 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:44:57.976875 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:44:57.976918 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:44:57.981199 (+ 4281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:57.982197 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:57.982205 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:57.982211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:57.982524 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:44:57.983029 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:44:57.983036 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:44:57.983039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:44:57.983339 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:44:57.983771 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0808 23:44:57.984039 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 23:44:57.984277 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:44:57.990361 9302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:44:58.001361 9302 catalog_manager.cc:1347] Generated new cluster ID: 518e032ad3394db8845636990f75406d
I20260808 23:44:58.001482 9302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:44:58.013245 9302 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:44:58.014586 9302 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:44:58.031010 9302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 730f782e545a4f329627d50383ccbc20: Generated new TSK 0
I20260808 23:44:58.031905 9302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:44:58.040072 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9212
I20260808 23:44:58.041626 9278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:44:58.152122 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 9212
2026-08-08T23:44:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2132 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:44:58 dist-test-slave-5k54 krb5kdc[9319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:44:58 dist-test-slave-5k54 krb5kdc[9319](info): set up 2 sockets
Aug 08 23:44:58 dist-test-slave-5k54 krb5kdc[9319](info): commencing operation
krb5kdc: starting...
W20260808 23:44:59.321955 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:44:59 dist-test-slave-5k54 krb5kdc[9319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:44:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:44:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:44:59.631666 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:44:59.972384 9333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:44:59.972702 9333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:44:59.972860 9333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:44:59.980170 9333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:44:59.980307 9333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:44:59.980417 9333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:44:59.980528 9333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:44:59.980604 9333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:44:59.990020 9333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:44:59.991308 9333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:44:59.992563 9333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:00.004455 9340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:00.004516 9342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:00.004520 9339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:00.006069 9333 server_base.cc:1034] running on GCE node
I20260808 23:45:00.007315 9333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:00.008973 9333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:00.010223 9333 hybrid_clock.cc:648] HybridClock initialized: now 1786232700010162 us; error 57 us; skew 500 ppm
I20260808 23:45:00.010601 9333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:00 dist-test-slave-5k54 krb5kdc[9319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232700, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:00.021274 9333 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:45:00.022931 9333 webserver.cc:460] Webserver started at http://127.0.0.1:44777/ using document root <none> and password file <none>
I20260808 23:45:00.023715 9333 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:00.023837 9333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:00.024108 9333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:00.027190 9333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "c128f4e80d134db5a2a5d57a9106519e"
format_stamp: "Formatted at 2026-08-08 23:45:00 on dist-test-slave-5k54"
server_key: "523c35d715bfe346cc76ad7f113d739f"
server_key_iv: "60ed136b527749a57365251c5aadb6c7"
server_key_version: "encryptionkey@0"
I20260808 23:45:00.027994 9333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "c128f4e80d134db5a2a5d57a9106519e"
format_stamp: "Formatted at 2026-08-08 23:45:00 on dist-test-slave-5k54"
server_key: "523c35d715bfe346cc76ad7f113d739f"
server_key_iv: "60ed136b527749a57365251c5aadb6c7"
server_key_version: "encryptionkey@0"
I20260808 23:45:00.033644 9333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:45:00.037361 9349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:00.038854 9333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:45:00.039013 9333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "c128f4e80d134db5a2a5d57a9106519e"
format_stamp: "Formatted at 2026-08-08 23:45:00 on dist-test-slave-5k54"
server_key: "523c35d715bfe346cc76ad7f113d739f"
server_key_iv: "60ed136b527749a57365251c5aadb6c7"
server_key_version: "encryptionkey@0"
I20260808 23:45:00.039196 9333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:00.061337 9333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:00.065943 9333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:00.066205 9333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:00.085764 9333 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33497
I20260808 23:45:00.085799 9406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33497 every 8 connection(s)
I20260808 23:45:00.087157 9333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:00.091022 9407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:00.091554 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 9333
I20260808 23:45:00.091737 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:00.092180 19178 external_mini_cluster.cc:1428] Setting key 78161ffd3f95c96ce65c87553b1759b5
I20260808 23:45:00.105739 9407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e: Bootstrap starting.
I20260808 23:45:00.107626 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:00.094111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55793 (local address 127.0.0.1:33497)
0808 23:45:00.094918 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:00.094933 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:00.094987 (+ 54us) server_negotiation.cc:407] Connection header received
0808 23:45:00.097680 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:00.097708 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:00.098226 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:00.098662 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:00.099171 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:00.101045 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:00.102353 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:00.103479 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:00.103666 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:00.104825 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:00.104836 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:00.104853 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:00.105582 (+ 729us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:00.105679 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:00.105711 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:00.105836 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:00.106057 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 23:45:00.106323 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":326,"threads_started":1}
I20260808 23:45:00.109361 9407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:00.110632 9407 log.cc:824] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:00.111307 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:00.113411 9407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e: No bootstrap required, opened a new log
I20260808 23:45:00.122328 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:00.115874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59589 (local address 127.0.0.1:33497)
0808 23:45:00.116089 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:00.116095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:00.116422 (+ 327us) server_negotiation.cc:407] Connection header received
0808 23:45:00.116567 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:00.116572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:00.116646 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:00.116761 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:00.117225 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:00.118301 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:00.119634 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:00.120357 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:00.120478 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:00.121585 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:00.121592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:00.121597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:00.121618 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:00.121669 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:00.121688 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:00.121775 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:00.121911 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 23:45:00.122009 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:45:00.123009 9407 raft_consensus.cc:348] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128f4e80d134db5a2a5d57a9106519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33497 } }
I20260808 23:45:00.123401 9407 raft_consensus.cc:374] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:00.123558 9407 raft_consensus.cc:729] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c128f4e80d134db5a2a5d57a9106519e, State: Initialized, Role: FOLLOWER
I20260808 23:45:00.124168 9407 consensus_queue.cc:260] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128f4e80d134db5a2a5d57a9106519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33497 } }
I20260808 23:45:00.124440 9407 raft_consensus.cc:388] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:00.124588 9407 raft_consensus.cc:482] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:00.124788 9407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:00.127013 9407 raft_consensus.cc:504] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128f4e80d134db5a2a5d57a9106519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33497 } }
I20260808 23:45:00.127615 9407 leader_election.cc:302] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c128f4e80d134db5a2a5d57a9106519e; no voters:
I20260808 23:45:00.128199 9407 leader_election.cc:288] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:00.128396 9417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:00.129935 9417 raft_consensus.cc:686] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [term 1 LEADER]: Becoming Leader. State: Replica: c128f4e80d134db5a2a5d57a9106519e, State: Running, Role: LEADER
I20260808 23:45:00.130491 9417 consensus_queue.cc:237] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128f4e80d134db5a2a5d57a9106519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33497 } }
I20260808 23:45:00.130617 9407 sys_catalog.cc:564] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:00.133405 9419 sys_catalog.cc:455] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c128f4e80d134db5a2a5d57a9106519e. Latest consensus state: current_term: 1 leader_uuid: "c128f4e80d134db5a2a5d57a9106519e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128f4e80d134db5a2a5d57a9106519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33497 } } }
I20260808 23:45:00.133765 9419 sys_catalog.cc:458] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:00.133912 9418 sys_catalog.cc:455] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c128f4e80d134db5a2a5d57a9106519e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128f4e80d134db5a2a5d57a9106519e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33497 } } }
I20260808 23:45:00.134313 9418 sys_catalog.cc:458] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:00.135408 9426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:00.140703 9426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:00.149868 9426 catalog_manager.cc:1347] Generated new cluster ID: bb27e84d088d496a830d0173a7587216
I20260808 23:45:00.150079 9426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:00.160897 9426 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:00.162228 9426 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:00.177871 9426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c128f4e80d134db5a2a5d57a9106519e: Generated new TSK 0
I20260808 23:45:00.178716 9426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:00.191672 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9333
W20260808 23:45:00.192720 9402 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55793
W20260808 23:45:00.193387 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9333: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:00.193531 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 9333
2026-08-08T23:45:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:00 dist-test-slave-5k54 krb5kdc[9438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:00 dist-test-slave-5k54 krb5kdc[9438](info): set up 2 sockets
Aug 08 23:45:00 dist-test-slave-5k54 krb5kdc[9438](info): commencing operation
krb5kdc: starting...
W20260808 23:45:01.354285 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:01 dist-test-slave-5k54 krb5kdc[9438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:45:01.675160 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:02.016300 9451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:02.016642 9451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:02.016820 9451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:02.023964 9451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:02.024124 9451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:02.024236 9451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:02.024349 9451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:02.024438 9451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:02.034140 9451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:02.035486 9451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:02.036854 9451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:02.048911 9457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:02.048942 9458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:02.049134 9460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:02.049716 9451 server_base.cc:1034] running on GCE node
I20260808 23:45:02.050499 9451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:02.051983 9451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:02.053203 9451 hybrid_clock.cc:648] HybridClock initialized: now 1786232702053155 us; error 39 us; skew 500 ppm
I20260808 23:45:02.053576 9451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:02 dist-test-slave-5k54 krb5kdc[9438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232702, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:02.056869 9451 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:45:02.058303 9451 webserver.cc:460] Webserver started at http://127.0.0.1:42777/ using document root <none> and password file <none>
I20260808 23:45:02.058991 9451 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:02.059128 9451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:02.059499 9451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:02.062597 9451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "ed776c57c30c4d6eb17a3c9eef79c92a"
format_stamp: "Formatted at 2026-08-08 23:45:02 on dist-test-slave-5k54"
server_key: "6c6b8c9b23184b1eca11d98c339da0ae"
server_key_iv: "e7b668c6afe254a4b26ca1ce924331d6"
server_key_version: "encryptionkey@0"
I20260808 23:45:02.063410 9451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "ed776c57c30c4d6eb17a3c9eef79c92a"
format_stamp: "Formatted at 2026-08-08 23:45:02 on dist-test-slave-5k54"
server_key: "6c6b8c9b23184b1eca11d98c339da0ae"
server_key_iv: "e7b668c6afe254a4b26ca1ce924331d6"
server_key_version: "encryptionkey@0"
I20260808 23:45:02.068979 9451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 23:45:02.072609 9468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:02.073977 9451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:45:02.074168 9451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "ed776c57c30c4d6eb17a3c9eef79c92a"
format_stamp: "Formatted at 2026-08-08 23:45:02 on dist-test-slave-5k54"
server_key: "6c6b8c9b23184b1eca11d98c339da0ae"
server_key_iv: "e7b668c6afe254a4b26ca1ce924331d6"
server_key_version: "encryptionkey@0"
I20260808 23:45:02.074376 9451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:02.080497 9451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:02.084558 9451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:02.084872 9451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:02.104323 9451 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34361
I20260808 23:45:02.104370 9523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34361 every 8 connection(s)
I20260808 23:45:02.105774 9451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:02.109580 9524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:02.115962 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 9451
I20260808 23:45:02.116164 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:02.116626 19178 external_mini_cluster.cc:1428] Setting key 4641a6b109326134e03bf3a619b78a84
I20260808 23:45:02.120074 9524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a: Bootstrap starting.
I20260808 23:45:02.123525 9524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:02.124620 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:02.118555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51243 (local address 127.0.0.1:34361)
0808 23:45:02.119370 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:02.119382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:02.119466 (+ 84us) server_negotiation.cc:407] Connection header received
0808 23:45:02.121216 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:02.121236 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:02.121578 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:02.121903 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:02.123014 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:02.123021 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:02.123033 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:02.123053 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:02.123106 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:02.123125 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:02.123264 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:02.123502 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 23:45:02.123724 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":369,"threads_started":1}
I20260808 23:45:02.124617 9524 log.cc:824] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:02.127655 9524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a: No bootstrap required, opened a new log
I20260808 23:45:02.128500 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:02.135363 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:02.132747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54169 (local address 127.0.0.1:34361)
0808 23:45:02.132911 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:02.132915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:02.133354 (+ 439us) server_negotiation.cc:407] Connection header received
0808 23:45:02.133596 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:02.133601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:02.133676 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:02.133795 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:02.134733 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:02.134749 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:02.134753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:02.134770 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:02.134808 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:02.134823 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:02.134966 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:02.135040 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 23:45:02.135098 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:45:02.136293 9524 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed776c57c30c4d6eb17a3c9eef79c92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34361 } }
I20260808 23:45:02.136734 9524 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:02.136827 9524 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed776c57c30c4d6eb17a3c9eef79c92a, State: Initialized, Role: FOLLOWER
I20260808 23:45:02.137458 9524 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed776c57c30c4d6eb17a3c9eef79c92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34361 } }
I20260808 23:45:02.137701 9524 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:02.137850 9524 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:02.138019 9524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:02.140470 9524 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed776c57c30c4d6eb17a3c9eef79c92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34361 } }
I20260808 23:45:02.140955 9524 leader_election.cc:302] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed776c57c30c4d6eb17a3c9eef79c92a; no voters:
I20260808 23:45:02.141516 9524 leader_election.cc:288] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:02.141674 9535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:02.143796 9524 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:02.143740 9535 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [term 1 LEADER]: Becoming Leader. State: Replica: ed776c57c30c4d6eb17a3c9eef79c92a, State: Running, Role: LEADER
I20260808 23:45:02.144392 9535 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed776c57c30c4d6eb17a3c9eef79c92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34361 } }
I20260808 23:45:02.147359 9537 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed776c57c30c4d6eb17a3c9eef79c92a. Latest consensus state: current_term: 1 leader_uuid: "ed776c57c30c4d6eb17a3c9eef79c92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed776c57c30c4d6eb17a3c9eef79c92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34361 } } }
I20260808 23:45:02.147881 9537 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:02.148465 9536 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed776c57c30c4d6eb17a3c9eef79c92a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed776c57c30c4d6eb17a3c9eef79c92a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34361 } } }
I20260808 23:45:02.148782 9536 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:02.150171 9547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:02.154399 9547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:02.162214 9547 catalog_manager.cc:1347] Generated new cluster ID: 60c85a252bff49a287ba2a3768714563
I20260808 23:45:02.162336 9547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:02.181433 9547 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:02.181882 9547 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:02.187286 9547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed776c57c30c4d6eb17a3c9eef79c92a: Generated new TSK 0
I20260808 23:45:02.188179 9547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:02.201611 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9451
W20260808 23:45:02.203143 9519 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51243
W20260808 23:45:02.203994 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9451: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:02.204155 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 9451
2026-08-08T23:45:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2010 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:02 dist-test-slave-5k54 krb5kdc[9556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:02 dist-test-slave-5k54 krb5kdc[9556](info): set up 2 sockets
Aug 08 23:45:02 dist-test-slave-5k54 krb5kdc[9556](info): commencing operation
krb5kdc: starting...
W20260808 23:45:03.370211 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:03 dist-test-slave-5k54 krb5kdc[9556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:45:03.694787 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:04.037945 9571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:04.038226 9571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:04.038321 9571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:04.045559 9571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:04.045670 9571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:04.045717 9571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:04.045779 9571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:04.045825 9571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:04.055575 9571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:04.056798 9571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:04.058089 9571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:04.069841 9576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:04.070011 9577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:04.070101 9579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:04.071689 9571 server_base.cc:1034] running on GCE node
I20260808 23:45:04.072731 9571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:04.074345 9571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:04.075572 9571 hybrid_clock.cc:648] HybridClock initialized: now 1786232704075512 us; error 58 us; skew 500 ppm
I20260808 23:45:04.075935 9571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:04 dist-test-slave-5k54 krb5kdc[9556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232704, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:04.079355 9571 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:45:04.081027 9571 webserver.cc:460] Webserver started at http://127.0.0.1:42137/ using document root <none> and password file <none>
I20260808 23:45:04.081766 9571 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:04.081933 9571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:04.082267 9571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:04.085757 9571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "b178968beb7d478fbb2cc9bab4366bca"
format_stamp: "Formatted at 2026-08-08 23:45:04 on dist-test-slave-5k54"
server_key: "3d969a1735481a7552040a3ab1565e95"
server_key_iv: "7e9aa336f6044bb98727097b2172b1d6"
server_key_version: "encryptionkey@0"
I20260808 23:45:04.086596 9571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "b178968beb7d478fbb2cc9bab4366bca"
format_stamp: "Formatted at 2026-08-08 23:45:04 on dist-test-slave-5k54"
server_key: "3d969a1735481a7552040a3ab1565e95"
server_key_iv: "7e9aa336f6044bb98727097b2172b1d6"
server_key_version: "encryptionkey@0"
I20260808 23:45:04.092311 9571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 23:45:04.096117 9586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:04.097502 9571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:45:04.097703 9571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "b178968beb7d478fbb2cc9bab4366bca"
format_stamp: "Formatted at 2026-08-08 23:45:04 on dist-test-slave-5k54"
server_key: "3d969a1735481a7552040a3ab1565e95"
server_key_iv: "7e9aa336f6044bb98727097b2172b1d6"
server_key_version: "encryptionkey@0"
I20260808 23:45:04.097932 9571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:04.115424 9571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:04.120005 9571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:04.120332 9571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:04.139379 9571 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35503
I20260808 23:45:04.139446 9639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35503 every 8 connection(s)
I20260808 23:45:04.140810 9571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:04.144598 9640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:04.146317 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 9571
I20260808 23:45:04.146517 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:04.147074 19178 external_mini_cluster.cc:1428] Setting key 17bcb03d1f62305f782e20109b7c74bf
I20260808 23:45:04.158938 9640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca: Bootstrap starting.
Aug 08 23:45:04 dist-test-slave-5k54 krb5kdc[9556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:45:04.162820 9640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:04.164023 9640 log.cc:824] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:04.166896 9640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca: No bootstrap required, opened a new log
I20260808 23:45:04.170781 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:04.148961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:43377 (local address 127.0.0.1:35503)
0808 23:45:04.149777 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:04.149788 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:04.149832 (+ 44us) server_negotiation.cc:407] Connection header received
0808 23:45:04.151489 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:04.151515 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:04.151919 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:04.152236 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:04.154039 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:04.155515 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:04.157337 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:04.158021 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:04.158155 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:04.163175 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:04.163186 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:04.163200 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:04.163239 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:04.166443 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:04.167338 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:04.167345 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:04.167351 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:04.167576 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:04.168057 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:04.168062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:04.168065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:04.168596 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:04.168972 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:04.169482 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 23:45:04.169807 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":603,"thread_start_us":387,"threads_started":1}
I20260808 23:45:04.176291 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:04.176342 9640 raft_consensus.cc:348] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b178968beb7d478fbb2cc9bab4366bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } }
I20260808 23:45:04.176743 9640 raft_consensus.cc:374] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:04.176827 9640 raft_consensus.cc:729] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b178968beb7d478fbb2cc9bab4366bca, State: Initialized, Role: FOLLOWER
I20260808 23:45:04.177451 9640 consensus_queue.cc:260] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b178968beb7d478fbb2cc9bab4366bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } }
I20260808 23:45:04.177714 9640 raft_consensus.cc:388] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:04.177830 9640 raft_consensus.cc:482] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:04.177985 9640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:04.181380 9640 raft_consensus.cc:504] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b178968beb7d478fbb2cc9bab4366bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } }
I20260808 23:45:04.181954 9640 leader_election.cc:302] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b178968beb7d478fbb2cc9bab4366bca; no voters:
I20260808 23:45:04.183197 9640 leader_election.cc:288] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:04.183296 9651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:04.184123 9651 raft_consensus.cc:686] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [term 1 LEADER]: Becoming Leader. State: Replica: b178968beb7d478fbb2cc9bab4366bca, State: Running, Role: LEADER
I20260808 23:45:04.185146 9651 consensus_queue.cc:237] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b178968beb7d478fbb2cc9bab4366bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } }
I20260808 23:45:04.187599 9651 sys_catalog.cc:455] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b178968beb7d478fbb2cc9bab4366bca. Latest consensus state: current_term: 1 leader_uuid: "b178968beb7d478fbb2cc9bab4366bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b178968beb7d478fbb2cc9bab4366bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } } }
I20260808 23:45:04.188002 9651 sys_catalog.cc:458] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:04.188356 9654 sys_catalog.cc:455] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b178968beb7d478fbb2cc9bab4366bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b178968beb7d478fbb2cc9bab4366bca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35503 } } }
I20260808 23:45:04.188782 9654 sys_catalog.cc:458] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:04.192056 9640 sys_catalog.cc:564] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:04.197947 9656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:04.201203 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:04.182624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:37667 (local address 127.0.0.1:35503)
0808 23:45:04.182885 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:04.182891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:04.183192 (+ 301us) server_negotiation.cc:407] Connection header received
0808 23:45:04.183400 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:04.183408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:04.183521 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:04.183631 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:04.185847 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:04.186607 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:04.188449 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:04.189170 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:04.189285 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:04.193335 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:04.193345 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:04.193353 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:04.193409 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:04.197747 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:04.198478 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:04.198485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:04.198493 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:04.198624 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:04.199172 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:04.199178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:04.199183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:04.199664 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:04.199929 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:04.200659 (+ 730us) server_negotiation.cc:299] Negotiation successful
0808 23:45:04.200868 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 23:45:04.205044 9656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:04.217566 9656 catalog_manager.cc:1347] Generated new cluster ID: e5e93bd7704b43ec8057202c8479ba57
I20260808 23:45:04.217695 9656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:04.229979 9656 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:04.231268 9656 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:04.239125 9656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b178968beb7d478fbb2cc9bab4366bca: Generated new TSK 0
I20260808 23:45:04.239996 9656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:04.246717 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9571
I20260808 23:45:04.248063 9633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:04.337198 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 9571
W20260808 23:45:04.358412 9554 connection.cc:535] client connection to 127.0.0.1:35503 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35503): Connection reset by peer (error 104)
2026-08-08T23:45:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2133 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:04 dist-test-slave-5k54 krb5kdc[9677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:04 dist-test-slave-5k54 krb5kdc[9677](info): set up 2 sockets
Aug 08 23:45:04 dist-test-slave-5k54 krb5kdc[9677](info): commencing operation
krb5kdc: starting...
W20260808 23:45:05.498576 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:05 dist-test-slave-5k54 krb5kdc[9677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:45:05.812960 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:06.149232 9690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:06.149578 9690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:06.149755 9690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:06.157002 9690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:06.157155 9690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:06.157279 9690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:06.157382 9690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:06.157467 9690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:06.167990 9690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:06.169286 9690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:06.170616 9690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:06.182583 9695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:06.182608 9698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:06.182930 9696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:06.183409 9690 server_base.cc:1034] running on GCE node
I20260808 23:45:06.184176 9690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:06.185534 9690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:06.186731 9690 hybrid_clock.cc:648] HybridClock initialized: now 1786232706186655 us; error 71 us; skew 500 ppm
I20260808 23:45:06.187083 9690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:06 dist-test-slave-5k54 krb5kdc[9677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232706, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:06.194873 9690 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:45:06.196370 9690 webserver.cc:460] Webserver started at http://127.0.0.1:42415/ using document root <none> and password file <none>
I20260808 23:45:06.196985 9690 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:06.197085 9690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:06.197346 9690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:06.200322 9690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "b36270d06437433cb58cc37dc18f14a7"
format_stamp: "Formatted at 2026-08-08 23:45:06 on dist-test-slave-5k54"
server_key: "c455bc9811bfc2bdb0eaba16f8622148"
server_key_iv: "8dd06f7525dc82cc79cd2221fa3169d3"
server_key_version: "encryptionkey@0"
I20260808 23:45:06.201016 9690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "b36270d06437433cb58cc37dc18f14a7"
format_stamp: "Formatted at 2026-08-08 23:45:06 on dist-test-slave-5k54"
server_key: "c455bc9811bfc2bdb0eaba16f8622148"
server_key_iv: "8dd06f7525dc82cc79cd2221fa3169d3"
server_key_version: "encryptionkey@0"
I20260808 23:45:06.206113 9690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:45:06.209434 9705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:06.210714 9690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:45:06.210877 9690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "b36270d06437433cb58cc37dc18f14a7"
format_stamp: "Formatted at 2026-08-08 23:45:06 on dist-test-slave-5k54"
server_key: "c455bc9811bfc2bdb0eaba16f8622148"
server_key_iv: "8dd06f7525dc82cc79cd2221fa3169d3"
server_key_version: "encryptionkey@0"
I20260808 23:45:06.211071 9690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:06.223541 9690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:06.227655 9690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:06.227942 9690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:06.247001 9690 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33919
I20260808 23:45:06.247048 9760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33919 every 8 connection(s)
I20260808 23:45:06.248394 9690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:06.252099 9763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:06.253634 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 9690
I20260808 23:45:06.253850 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:06.254438 19178 external_mini_cluster.cc:1428] Setting key ee7f96b23b95e8979ac0903cd2480b62
I20260808 23:45:06.266332 9763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7: Bootstrap starting.
I20260808 23:45:06.267848 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:06.256271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:52801 (local address 127.0.0.1:33919)
0808 23:45:06.257043 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:06.257056 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:06.257095 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:45:06.258713 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:06.258736 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:06.259144 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:06.259534 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:06.260162 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:06.261488 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:06.263384 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:06.264119 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:06.264240 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:06.265547 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:06.265554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:06.265566 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:06.266078 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:06.266149 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:06.266172 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:06.266284 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:06.266646 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 23:45:06.266838 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":385,"threads_started":1}
I20260808 23:45:06.269589 9763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:06.270749 9763 log.cc:824] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:06.271486 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:06.273404 9763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7: No bootstrap required, opened a new log
I20260808 23:45:06.282667 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:06.275915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:41439 (local address 127.0.0.1:33919)
0808 23:45:06.276065 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:06.276069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:06.276622 (+ 553us) server_negotiation.cc:407] Connection header received
0808 23:45:06.276787 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:06.276791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:06.276852 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:06.276926 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:06.277500 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:06.278247 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:06.280120 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:06.280543 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:06.280635 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:06.281885 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:06.281890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:06.281893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:06.281911 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:06.281946 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:06.281964 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:06.282058 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:06.282270 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 23:45:06.282345 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 23:45:06.283058 9763 raft_consensus.cc:348] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36270d06437433cb58cc37dc18f14a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33919 } }
I20260808 23:45:06.283514 9763 raft_consensus.cc:374] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:06.283628 9763 raft_consensus.cc:729] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36270d06437433cb58cc37dc18f14a7, State: Initialized, Role: FOLLOWER
I20260808 23:45:06.284171 9763 consensus_queue.cc:260] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36270d06437433cb58cc37dc18f14a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33919 } }
I20260808 23:45:06.284417 9763 raft_consensus.cc:388] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:06.284555 9763 raft_consensus.cc:482] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:06.284722 9763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:06.286871 9763 raft_consensus.cc:504] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36270d06437433cb58cc37dc18f14a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33919 } }
I20260808 23:45:06.287341 9763 leader_election.cc:302] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b36270d06437433cb58cc37dc18f14a7; no voters:
I20260808 23:45:06.287954 9763 leader_election.cc:288] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:06.288089 9773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:06.289995 9763 sys_catalog.cc:564] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:06.290045 9773 raft_consensus.cc:686] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [term 1 LEADER]: Becoming Leader. State: Replica: b36270d06437433cb58cc37dc18f14a7, State: Running, Role: LEADER
I20260808 23:45:06.290609 9773 consensus_queue.cc:237] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36270d06437433cb58cc37dc18f14a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33919 } }
I20260808 23:45:06.293812 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b36270d06437433cb58cc37dc18f14a7. Latest consensus state: current_term: 1 leader_uuid: "b36270d06437433cb58cc37dc18f14a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36270d06437433cb58cc37dc18f14a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33919 } } }
I20260808 23:45:06.294174 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:06.294898 9775 sys_catalog.cc:455] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b36270d06437433cb58cc37dc18f14a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36270d06437433cb58cc37dc18f14a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33919 } } }
I20260808 23:45:06.295248 9775 sys_catalog.cc:458] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:06.298012 9785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:06.302776 9785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:06.310838 9785 catalog_manager.cc:1347] Generated new cluster ID: be2b9b1ef77e40a8a7dfbd10e25f7440
I20260808 23:45:06.310962 9785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:06.325507 9785 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:06.326771 9785 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:06.343510 9785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b36270d06437433cb58cc37dc18f14a7: Generated new TSK 0
I20260808 23:45:06.344444 9785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:06.362020 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9690
W20260808 23:45:06.363163 9755 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.29:52801
W20260808 23:45:06.364027 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9690: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:06.364168 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 9690
W20260808 23:45:06.384622 9675 connection.cc:535] client connection to 127.0.0.1:33919 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33919): Connection reset by peer (error 104)
2026-08-08T23:45:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2026 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:06 dist-test-slave-5k54 krb5kdc[9797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:06 dist-test-slave-5k54 krb5kdc[9797](info): set up 2 sockets
Aug 08 23:45:06 dist-test-slave-5k54 krb5kdc[9797](info): commencing operation
krb5kdc: starting...
W20260808 23:45:07.562860 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:07 dist-test-slave-5k54 krb5kdc[9797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:45:07.886720 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44331
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:08.241528 9811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:08.241815 9811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:08.241992 9811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:08.249141 9811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:08.249289 9811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:08.249406 9811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:08.249511 9811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:08.249593 9811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:08.259287 9811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:08.260596 9811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:08.261911 9811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:08.274080 9817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:08.274133 9820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:08.274114 9818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:08.274456 9811 server_base.cc:1034] running on GCE node
I20260808 23:45:08.275610 9811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:08.276892 9811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:08.278096 9811 hybrid_clock.cc:648] HybridClock initialized: now 1786232708278056 us; error 25 us; skew 500 ppm
I20260808 23:45:08.278404 9811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:08 dist-test-slave-5k54 krb5kdc[9797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232708, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:08.281592 9811 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:45:08.282979 9811 webserver.cc:460] Webserver started at http://127.0.0.1:37205/ using document root <none> and password file <none>
I20260808 23:45:08.283674 9811 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:08.283800 9811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:08.284052 9811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:08.287099 9811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "82e7ac3e7b034700971d161861c913de"
format_stamp: "Formatted at 2026-08-08 23:45:08 on dist-test-slave-5k54"
server_key: "f2b9eabdce5723a656d8898be31696c4"
server_key_iv: "cea8bc4d7b182ec36362a9618e6d7180"
server_key_version: "encryptionkey@0"
I20260808 23:45:08.287894 9811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "82e7ac3e7b034700971d161861c913de"
format_stamp: "Formatted at 2026-08-08 23:45:08 on dist-test-slave-5k54"
server_key: "f2b9eabdce5723a656d8898be31696c4"
server_key_iv: "cea8bc4d7b182ec36362a9618e6d7180"
server_key_version: "encryptionkey@0"
I20260808 23:45:08.292986 9811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:45:08.296561 9827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:08.297749 9811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:45:08.297935 9811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "82e7ac3e7b034700971d161861c913de"
format_stamp: "Formatted at 2026-08-08 23:45:08 on dist-test-slave-5k54"
server_key: "f2b9eabdce5723a656d8898be31696c4"
server_key_iv: "cea8bc4d7b182ec36362a9618e6d7180"
server_key_version: "encryptionkey@0"
I20260808 23:45:08.298131 9811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:08.303684 9811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:08.308002 9811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:08.308295 9811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:08.326651 9811 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44331
I20260808 23:45:08.326736 9880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44331 every 8 connection(s)
I20260808 23:45:08.328045 9811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:08.331669 9881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:08.336645 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 9811
I20260808 23:45:08.336845 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:08.337404 19178 external_mini_cluster.cc:1428] Setting key d893c097e47d098c7cf2a3a1c93cbcee
I20260808 23:45:08.343297 9881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de: Bootstrap starting.
I20260808 23:45:08.345325 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:08.339320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:38777 (local address 127.0.0.1:44331)
0808 23:45:08.340126 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:08.340141 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:08.340180 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:45:08.341942 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:08.341963 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:08.342281 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:08.342661 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:08.343719 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:08.343725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:08.343743 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:08.343758 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:08.343820 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:08.343842 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:08.344083 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:08.344216 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 23:45:08.344392 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":332,"threads_started":1}
I20260808 23:45:08.346606 9881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:08.347824 9881 log.cc:824] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:08.349133 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:08.350692 9881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de: No bootstrap required, opened a new log
I20260808 23:45:08.356175 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:08.353704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:53909 (local address 127.0.0.1:44331)
0808 23:45:08.353875 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:08.353881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:08.354235 (+ 354us) server_negotiation.cc:407] Connection header received
0808 23:45:08.354364 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:08.354369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:08.354424 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:08.354541 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:08.355487 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:08.355492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:08.355497 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:08.355516 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:08.355567 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:08.355591 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:08.355727 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:08.355826 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 23:45:08.355887 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:45:08.359102 9881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e7ac3e7b034700971d161861c913de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44331 } }
I20260808 23:45:08.359508 9881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:08.359651 9881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e7ac3e7b034700971d161861c913de, State: Initialized, Role: FOLLOWER
I20260808 23:45:08.360212 9881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e7ac3e7b034700971d161861c913de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44331 } }
I20260808 23:45:08.360453 9881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:08.360603 9881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:08.360770 9881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:08.363235 9881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e7ac3e7b034700971d161861c913de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44331 } }
I20260808 23:45:08.363765 9881 leader_election.cc:302] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82e7ac3e7b034700971d161861c913de; no voters:
I20260808 23:45:08.364328 9881 leader_election.cc:288] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:08.364490 9891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:08.365170 9891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [term 1 LEADER]: Becoming Leader. State: Replica: 82e7ac3e7b034700971d161861c913de, State: Running, Role: LEADER
I20260808 23:45:08.365677 9891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e7ac3e7b034700971d161861c913de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44331 } }
I20260808 23:45:08.366369 9881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:08.368558 9893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e7ac3e7b034700971d161861c913de. Latest consensus state: current_term: 1 leader_uuid: "82e7ac3e7b034700971d161861c913de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e7ac3e7b034700971d161861c913de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44331 } } }
I20260808 23:45:08.368702 9892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82e7ac3e7b034700971d161861c913de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82e7ac3e7b034700971d161861c913de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44331 } } }
I20260808 23:45:08.369202 9892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:08.369208 9893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:08.372015 9896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:08.377079 9896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:08.385542 9896 catalog_manager.cc:1347] Generated new cluster ID: c02c66081f134dac8d454ebbe361f4a1
I20260808 23:45:08.385685 9896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:08.399509 9896 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:08.399932 9896 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:08.407907 9896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82e7ac3e7b034700971d161861c913de: Generated new TSK 0
I20260808 23:45:08.408777 9896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:08.424435 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9811
W20260808 23:45:08.425695 9876 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.29:38777
W20260808 23:45:08.426517 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9811: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:08.426689 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 9811
2026-08-08T23:45:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2064 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:08 dist-test-slave-5k54 krb5kdc[9912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:08 dist-test-slave-5k54 krb5kdc[9912](info): set up 2 sockets
Aug 08 23:45:08 dist-test-slave-5k54 krb5kdc[9912](info): commencing operation
krb5kdc: starting...
W20260808 23:45:09.605957 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:09 dist-test-slave-5k54 krb5kdc[9912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:45:09.926623 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:10.265273 9929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:10.265560 9929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:10.265748 9929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:10.272934 9929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:10.273082 9929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:10.273176 9929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:10.273294 9929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:10.273371 9929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:10.283042 9929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38649
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:10.284302 9929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:10.285676 9929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:10.297214 9935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:10.297279 9936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:10.297298 9938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:10.298375 9929 server_base.cc:1034] running on GCE node
I20260808 23:45:10.299171 9929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:10.300529 9929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:10.301733 9929 hybrid_clock.cc:648] HybridClock initialized: now 1786232710301692 us; error 32 us; skew 500 ppm
I20260808 23:45:10.302074 9929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:10 dist-test-slave-5k54 krb5kdc[9912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:10.305222 9929 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:45:10.306720 9929 webserver.cc:460] Webserver started at http://127.0.0.1:44811/ using document root <none> and password file <none>
I20260808 23:45:10.307382 9929 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:10.307560 9929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:10.307822 9929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:10.310690 9929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "f5790d0835cc491ca21bb437e238315b"
format_stamp: "Formatted at 2026-08-08 23:45:10 on dist-test-slave-5k54"
server_key: "4171e9f5dfe2839aa96a661a3bc61465"
server_key_iv: "8b065ffa6e99b47135588e865eb5c2eb"
server_key_version: "encryptionkey@0"
I20260808 23:45:10.311395 9929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "f5790d0835cc491ca21bb437e238315b"
format_stamp: "Formatted at 2026-08-08 23:45:10 on dist-test-slave-5k54"
server_key: "4171e9f5dfe2839aa96a661a3bc61465"
server_key_iv: "8b065ffa6e99b47135588e865eb5c2eb"
server_key_version: "encryptionkey@0"
I20260808 23:45:10.316339 9929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 23:45:10.319660 9946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:10.320962 9929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:45:10.321153 9929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "f5790d0835cc491ca21bb437e238315b"
format_stamp: "Formatted at 2026-08-08 23:45:10 on dist-test-slave-5k54"
server_key: "4171e9f5dfe2839aa96a661a3bc61465"
server_key_iv: "8b065ffa6e99b47135588e865eb5c2eb"
server_key_version: "encryptionkey@0"
I20260808 23:45:10.321341 9929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:10.335827 9929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:10.339707 9929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:10.339991 9929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:10.358489 9929 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38649
I20260808 23:45:10.358518 10001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38649 every 8 connection(s)
I20260808 23:45:10.359838 9929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:10.363389 10002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:10.365072 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 9929
I20260808 23:45:10.365265 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:10.365787 19178 external_mini_cluster.cc:1428] Setting key 6b5bc3dff5c8a9b083404c3011ec3e4f
I20260808 23:45:10.375854 10002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b: Bootstrap starting.
I20260808 23:45:10.379701 10002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:10.381006 10002 log.cc:824] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:45:10 dist-test-slave-5k54 krb5kdc[9912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:45:10.383966 10002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b: No bootstrap required, opened a new log
I20260808 23:45:10.390533 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:10.367621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:54935 (local address 127.0.0.1:38649)
0808 23:45:10.368981 (+ 1360us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:10.368992 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:10.369036 (+ 44us) server_negotiation.cc:407] Connection header received
0808 23:45:10.370693 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:10.370713 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:10.371101 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:10.371401 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:10.373440 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:10.374738 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:10.376571 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:10.377257 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:10.377394 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:10.382677 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:10.382685 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:10.382699 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:10.382739 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:10.386157 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:10.386977 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:10.386983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:10.386991 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:10.387162 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:10.387738 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:10.387742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:10.387744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:10.388284 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:10.388610 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:10.389209 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 23:45:10.389509 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1144,"thread_start_us":403,"threads_started":1}
I20260808 23:45:10.391903 10002 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5790d0835cc491ca21bb437e238315b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } }
I20260808 23:45:10.392238 10002 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:10.392386 10002 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5790d0835cc491ca21bb437e238315b, State: Initialized, Role: FOLLOWER
I20260808 23:45:10.392942 10002 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5790d0835cc491ca21bb437e238315b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } }
I20260808 23:45:10.393280 10002 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:10.393433 10002 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:10.393630 10002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:10.394603 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:10.396118 10002 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5790d0835cc491ca21bb437e238315b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } }
I20260808 23:45:10.396598 10002 leader_election.cc:302] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5790d0835cc491ca21bb437e238315b; no voters:
I20260808 23:45:10.397265 10002 leader_election.cc:288] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:10.397377 10007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:10.404837 10007 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [term 1 LEADER]: Becoming Leader. State: Replica: f5790d0835cc491ca21bb437e238315b, State: Running, Role: LEADER
I20260808 23:45:10.405519 10007 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5790d0835cc491ca21bb437e238315b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } }
I20260808 23:45:10.405758 10002 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:10.409348 10013 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5790d0835cc491ca21bb437e238315b. Latest consensus state: current_term: 1 leader_uuid: "f5790d0835cc491ca21bb437e238315b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5790d0835cc491ca21bb437e238315b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } } }
I20260808 23:45:10.409710 10013 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:10.410245 10011 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5790d0835cc491ca21bb437e238315b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5790d0835cc491ca21bb437e238315b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38649 } } }
I20260808 23:45:10.410549 10011 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:10.410646 10022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:10.427546 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:10.399225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:56767 (local address 127.0.0.1:38649)
0808 23:45:10.399507 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:10.399516 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:10.405769 (+ 6253us) server_negotiation.cc:407] Connection header received
0808 23:45:10.405983 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:10.405989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:10.406066 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:10.406168 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:10.407922 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:10.409269 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:10.411660 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:10.412655 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:10.412820 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:10.417002 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:10.417012 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:10.417020 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:10.417070 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:10.421970 (+ 4900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:10.423228 (+ 1258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:10.423237 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:10.423243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:10.423460 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:10.424082 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:10.424088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:10.424091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:10.424651 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:10.426158 (+ 1507us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:10.426360 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 23:45:10.426567 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:45:10.430816 10022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:10.440397 10022 catalog_manager.cc:1347] Generated new cluster ID: 1ac5002a8f274bd5bb448484bb853c8d
I20260808 23:45:10.440500 10022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:10.450722 10022 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:10.452112 10022 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:10.465101 10022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5790d0835cc491ca21bb437e238315b: Generated new TSK 0
I20260808 23:45:10.465893 10022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:10.473179 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 9929
I20260808 23:45:10.474267 9996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:10.565281 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 9929
W20260808 23:45:10.585901 9910 connection.cc:535] client connection to 127.0.0.1:38649 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38649): Connection reset by peer (error 104)
2026-08-08T23:45:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2137 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:10 dist-test-slave-5k54 krb5kdc[10037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:10 dist-test-slave-5k54 krb5kdc[10037](info): set up 2 sockets
Aug 08 23:45:10 dist-test-slave-5k54 krb5kdc[10037](info): commencing operation
krb5kdc: starting...
W20260808 23:45:11.744613 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:11 dist-test-slave-5k54 krb5kdc[10037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:45:12.075150 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:12.417435 10052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:12.417712 10052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:12.417896 10052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:12.425114 10052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:12.425220 10052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:12.425334 10052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:12.425418 10052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:12.425503 10052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:12.435384 10052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38533
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:12.436683 10052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:12.438022 10052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:12.450228 10058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:12.450263 10057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:12.450251 10060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:12.451311 10052 server_base.cc:1034] running on GCE node
I20260808 23:45:12.452135 10052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:12.453527 10052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:12.454764 10052 hybrid_clock.cc:648] HybridClock initialized: now 1786232712454731 us; error 58 us; skew 500 ppm
I20260808 23:45:12.455137 10052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:12 dist-test-slave-5k54 krb5kdc[10037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:12.463027 10052 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:45:12.464689 10052 webserver.cc:460] Webserver started at http://127.0.0.1:40863/ using document root <none> and password file <none>
I20260808 23:45:12.465431 10052 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:12.465588 10052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:12.465895 10052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:12.468940 10052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "85e0b04149154f0194cf90d0f5bdf2ff"
format_stamp: "Formatted at 2026-08-08 23:45:12 on dist-test-slave-5k54"
server_key: "06063e17b167592e54a0a401c86d625b"
server_key_iv: "676119ca05cf8c86e08f256745c2b15b"
server_key_version: "encryptionkey@0"
I20260808 23:45:12.469708 10052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "85e0b04149154f0194cf90d0f5bdf2ff"
format_stamp: "Formatted at 2026-08-08 23:45:12 on dist-test-slave-5k54"
server_key: "06063e17b167592e54a0a401c86d625b"
server_key_iv: "676119ca05cf8c86e08f256745c2b15b"
server_key_version: "encryptionkey@0"
I20260808 23:45:12.474898 10052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 23:45:12.478336 10069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:12.479732 10052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:45:12.479913 10052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "85e0b04149154f0194cf90d0f5bdf2ff"
format_stamp: "Formatted at 2026-08-08 23:45:12 on dist-test-slave-5k54"
server_key: "06063e17b167592e54a0a401c86d625b"
server_key_iv: "676119ca05cf8c86e08f256745c2b15b"
server_key_version: "encryptionkey@0"
I20260808 23:45:12.480109 10052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:12.511740 10052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:12.516263 10052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:12.516582 10052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:12.535707 10052 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38533
I20260808 23:45:12.535764 10124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38533 every 8 connection(s)
I20260808 23:45:12.537223 10052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:12.541123 10125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:12.547050 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 10052
I20260808 23:45:12.547251 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:12.547771 19178 external_mini_cluster.cc:1428] Setting key 2c2c143d9b4d73047e8a8e2be2474871
I20260808 23:45:12.552506 10125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff: Bootstrap starting.
I20260808 23:45:12.556077 10125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:12.557284 10125 log.cc:824] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:12.560691 10125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff: No bootstrap required, opened a new log
I20260808 23:45:12.561573 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:12.549707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:41845 (local address 127.0.0.1:38533)
0808 23:45:12.550597 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:12.550609 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:12.550651 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:45:12.552357 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:12.552381 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:12.552825 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:12.553151 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:12.553766 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:12.555071 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:12.556951 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:12.557682 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:12.557937 (+ 255us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:12.559225 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:12.559232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:12.559246 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:12.559833 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:12.559915 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:12.559940 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:12.560054 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:12.560363 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 23:45:12.560548 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":416,"threads_started":1}
I20260808 23:45:12.565496 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:12.568734 10125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e0b04149154f0194cf90d0f5bdf2ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } }
I20260808 23:45:12.569104 10125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:12.569204 10125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e0b04149154f0194cf90d0f5bdf2ff, State: Initialized, Role: FOLLOWER
I20260808 23:45:12.569868 10125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e0b04149154f0194cf90d0f5bdf2ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } }
I20260808 23:45:12.570210 10125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:12.570475 10125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:12.570669 10125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:12.573767 10125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e0b04149154f0194cf90d0f5bdf2ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } }
I20260808 23:45:12.574316 10125 leader_election.cc:302] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85e0b04149154f0194cf90d0f5bdf2ff; no voters:
I20260808 23:45:12.574996 10125 leader_election.cc:288] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:12.575098 10136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:12.575896 10136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [term 1 LEADER]: Becoming Leader. State: Replica: 85e0b04149154f0194cf90d0f5bdf2ff, State: Running, Role: LEADER
I20260808 23:45:12.576848 10136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e0b04149154f0194cf90d0f5bdf2ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } }
I20260808 23:45:12.577890 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:12.570116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:39079 (local address 127.0.0.1:38533)
0808 23:45:12.570359 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:12.570368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:12.570852 (+ 484us) server_negotiation.cc:407] Connection header received
0808 23:45:12.570989 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:12.570996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:12.571085 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:12.571194 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:12.571744 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:12.572936 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:12.574314 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:12.575139 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:12.575278 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:12.576653 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:12.576659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:12.576663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:12.576687 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:12.576757 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:12.576779 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:12.576884 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:12.577308 (+ 424us) server_negotiation.cc:299] Negotiation successful
0808 23:45:12.577396 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 23:45:12.579195 10136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85e0b04149154f0194cf90d0f5bdf2ff. Latest consensus state: current_term: 1 leader_uuid: "85e0b04149154f0194cf90d0f5bdf2ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e0b04149154f0194cf90d0f5bdf2ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } } }
I20260808 23:45:12.579573 10136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:12.581518 10138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85e0b04149154f0194cf90d0f5bdf2ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e0b04149154f0194cf90d0f5bdf2ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } } }
I20260808 23:45:12.581894 10138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:12.583773 10125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:12.584587 10140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:12.590063 10140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:12.600095 10140 catalog_manager.cc:1347] Generated new cluster ID: 4d1f0d4f95e349129768812e07ccec54
I20260808 23:45:12.600209 10140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:12.616657 10140 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:12.617905 10140 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:12.628322 10140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85e0b04149154f0194cf90d0f5bdf2ff: Generated new TSK 0
I20260808 23:45:12.629168 10140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:12.644273 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10052
W20260808 23:45:12.645462 10119 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.29:41845
W20260808 23:45:12.646267 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10052: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:12.646425 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 10052
W20260808 23:45:12.666939 10035 connection.cc:535] client connection to 127.0.0.1:38533 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38533): Connection reset by peer (error 104)
2026-08-08T23:45:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2085 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:12 dist-test-slave-5k54 krb5kdc[10158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:12 dist-test-slave-5k54 krb5kdc[10158](info): set up 2 sockets
Aug 08 23:45:12 dist-test-slave-5k54 krb5kdc[10158](info): commencing operation
krb5kdc: starting...
W20260808 23:45:13.814666 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:14 dist-test-slave-5k54 krb5kdc[10158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:45:14.131318 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:14.473416 10171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:14.473708 10171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:14.473803 10171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:14.481020 10171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:14.481137 10171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:14.481184 10171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:14.481245 10171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:14.481290 10171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:14.491297 10171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:14.492569 10171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:14.493798 10171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:14.505753 10177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:14.505841 10179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:14.505811 10176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:14.506749 10171 server_base.cc:1034] running on GCE node
I20260808 23:45:14.507681 10171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:14.509060 10171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:14.510299 10171 hybrid_clock.cc:648] HybridClock initialized: now 1786232714510257 us; error 54 us; skew 500 ppm
I20260808 23:45:14.510669 10171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:14 dist-test-slave-5k54 krb5kdc[10158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232714, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:14.513774 10171 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:45:14.515280 10171 webserver.cc:460] Webserver started at http://127.0.0.1:40073/ using document root <none> and password file <none>
I20260808 23:45:14.516024 10171 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:14.516175 10171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:14.516486 10171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:14.519515 10171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd"
format_stamp: "Formatted at 2026-08-08 23:45:14 on dist-test-slave-5k54"
server_key: "6d63c535ee8b3d3835ed48c2cdf7b9b9"
server_key_iv: "9ab9ca6e1a48ea63c0eb39a05ab87500"
server_key_version: "encryptionkey@0"
I20260808 23:45:14.520311 10171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd"
format_stamp: "Formatted at 2026-08-08 23:45:14 on dist-test-slave-5k54"
server_key: "6d63c535ee8b3d3835ed48c2cdf7b9b9"
server_key_iv: "9ab9ca6e1a48ea63c0eb39a05ab87500"
server_key_version: "encryptionkey@0"
I20260808 23:45:14.525447 10171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:45:14.528934 10188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:14.530187 10171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:45:14.530383 10171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd"
format_stamp: "Formatted at 2026-08-08 23:45:14 on dist-test-slave-5k54"
server_key: "6d63c535ee8b3d3835ed48c2cdf7b9b9"
server_key_iv: "9ab9ca6e1a48ea63c0eb39a05ab87500"
server_key_version: "encryptionkey@0"
I20260808 23:45:14.530583 10171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:14.536458 10171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:14.540462 10171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:14.540755 10171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:14.559173 10171 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38155
I20260808 23:45:14.559222 10245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38155 every 8 connection(s)
I20260808 23:45:14.560603 10171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:14.561659 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 10171
I20260808 23:45:14.561861 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:14.562445 19178 external_mini_cluster.cc:1428] Setting key 4749ef1fc4a117121fc762e8e7dd9393
I20260808 23:45:14.565253 10246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:14.570315 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:14.564340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:50575 (local address 127.0.0.1:38155)
0808 23:45:14.565169 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:14.565181 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:14.565222 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:45:14.566844 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:14.566864 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:14.567174 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:14.567635 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:14.568706 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:14.568712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:14.568725 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:14.568739 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:14.568821 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:14.568859 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:14.569037 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:14.569184 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 23:45:14.569393 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":387,"threads_started":1}
I20260808 23:45:14.573982 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:14.578087 10246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd: Bootstrap starting.
I20260808 23:45:14.581791 10246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:14.582217 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:14.579404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:43839 (local address 127.0.0.1:38155)
0808 23:45:14.579610 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:14.579618 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:14.580209 (+ 591us) server_negotiation.cc:407] Connection header received
0808 23:45:14.580386 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:14.580391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:14.580460 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:14.580569 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:14.581532 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:14.581538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:14.581542 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:14.581566 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:14.581616 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:14.581639 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:14.581770 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:14.581850 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 23:45:14.581913 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 23:45:14.583066 10246 log.cc:824] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:14.585937 10246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd: No bootstrap required, opened a new log
I20260808 23:45:14.593816 10246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38155 } }
I20260808 23:45:14.594228 10246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:14.594339 10246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b04ce2d5ddf4c22bca17ce6eadeeefd, State: Initialized, Role: FOLLOWER
I20260808 23:45:14.594914 10246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38155 } }
I20260808 23:45:14.595168 10246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:14.595321 10246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:14.595535 10246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:14.597932 10246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38155 } }
I20260808 23:45:14.598464 10246 leader_election.cc:302] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b04ce2d5ddf4c22bca17ce6eadeeefd; no voters:
I20260808 23:45:14.599072 10246 leader_election.cc:288] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:14.599254 10257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:14.600070 10257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [term 1 LEADER]: Becoming Leader. State: Replica: 4b04ce2d5ddf4c22bca17ce6eadeeefd, State: Running, Role: LEADER
I20260808 23:45:14.600669 10257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38155 } }
I20260808 23:45:14.601229 10246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:14.603953 10259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b04ce2d5ddf4c22bca17ce6eadeeefd. Latest consensus state: current_term: 1 leader_uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38155 } } }
I20260808 23:45:14.603803 10258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b04ce2d5ddf4c22bca17ce6eadeeefd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38155 } } }
I20260808 23:45:14.604651 10259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:14.604651 10258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:14.606699 10264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:14.611673 10264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:14.620220 10264 catalog_manager.cc:1347] Generated new cluster ID: 900cd13ebc12442ebf7fb88d4b0ddb94
I20260808 23:45:14.620754 10264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:14.638425 10264 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:14.638854 10264 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:14.652726 10264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b04ce2d5ddf4c22bca17ce6eadeeefd: Generated new TSK 0
I20260808 23:45:14.653573 10264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:14.661453 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10171
W20260808 23:45:14.663025 10240 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.29:50575
W20260808 23:45:14.663741 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10171: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:14.663914 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 10171
2026-08-08T23:45:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2013 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:14 dist-test-slave-5k54 krb5kdc[10281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:14 dist-test-slave-5k54 krb5kdc[10281](info): set up 2 sockets
Aug 08 23:45:14 dist-test-slave-5k54 krb5kdc[10281](info): commencing operation
krb5kdc: starting...
W20260808 23:45:15.850543 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:16 dist-test-slave-5k54 krb5kdc[10281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:45:16.162092 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44319
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:16.500171 10297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:16.500502 10297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:16.500676 10297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:16.507826 10297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:16.507977 10297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:16.508078 10297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:16.508183 10297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:16.508265 10297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:16.518309 10297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:44319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:16.519714 10297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:16.521025 10297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:16.533247 10305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:16.533309 10304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:16.533474 10307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:16.534132 10297 server_base.cc:1034] running on GCE node
I20260808 23:45:16.534991 10297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:16.536451 10297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:16.537650 10297 hybrid_clock.cc:648] HybridClock initialized: now 1786232716537595 us; error 55 us; skew 500 ppm
I20260808 23:45:16.538049 10297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:16 dist-test-slave-5k54 krb5kdc[10281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:16.541313 10297 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:45:16.542855 10297 webserver.cc:460] Webserver started at http://127.18.186.190:45391/ using document root <none> and password file <none>
I20260808 23:45:16.543613 10297 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:16.543746 10297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:16.544063 10297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:16.547051 10297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253"
format_stamp: "Formatted at 2026-08-08 23:45:16 on dist-test-slave-5k54"
server_key: "1e200c051af84bc7c0725573aae0eb88"
server_key_iv: "e58d903d77686abcea26d906aad2ac90"
server_key_version: "encryptionkey@0"
I20260808 23:45:16.547854 10297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253"
format_stamp: "Formatted at 2026-08-08 23:45:16 on dist-test-slave-5k54"
server_key: "1e200c051af84bc7c0725573aae0eb88"
server_key_iv: "e58d903d77686abcea26d906aad2ac90"
server_key_version: "encryptionkey@0"
I20260808 23:45:16.552989 10297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:45:16.556403 10314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:16.557665 10297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:45:16.557863 10297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253"
format_stamp: "Formatted at 2026-08-08 23:45:16 on dist-test-slave-5k54"
server_key: "1e200c051af84bc7c0725573aae0eb88"
server_key_iv: "e58d903d77686abcea26d906aad2ac90"
server_key_version: "encryptionkey@0"
I20260808 23:45:16.558054 10297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:16.604494 10297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:16.608971 10297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:16.609287 10297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:16.630214 10297 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:44319
I20260808 23:45:16.630234 10367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:44319 every 8 connection(s)
I20260808 23:45:16.631698 10297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:16.633026 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 10297
I20260808 23:45:16.633220 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:16.633800 19178 external_mini_cluster.cc:1428] Setting key 340a262f30d261edea587f5980cac1a2
I20260808 23:45:16.636546 10368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:45:16 dist-test-slave-5k54 krb5kdc[10281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:16.649794 10368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253: Bootstrap starting.
I20260808 23:45:16.652971 10368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:16.654075 10368 log.cc:824] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:16.657107 10368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253: No bootstrap required, opened a new log
I20260808 23:45:16.657903 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:16.635761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51219 (local address 127.18.186.190:44319)
0808 23:45:16.636670 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:16.636686 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:16.636742 (+ 56us) server_negotiation.cc:407] Connection header received
0808 23:45:16.638521 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:16.638542 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:16.638957 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:16.639324 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:16.641176 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:16.642719 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:16.644534 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:16.645290 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:16.645450 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:16.650387 (+ 4937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:16.650395 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:16.650409 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:16.650449 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:16.653625 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:16.654475 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:16.654481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:16.654487 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:16.654659 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:16.655171 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:16.655174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:16.655176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:16.655740 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:16.656058 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:16.656576 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 23:45:16.656879 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":387,"threads_started":1}
I20260808 23:45:16.661818 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:16.665357 10368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44319 } }
I20260808 23:45:16.665727 10368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:16.665845 10368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cee43e9b8dd4ffd9f23622d3a6b6253, State: Initialized, Role: FOLLOWER
I20260808 23:45:16.666536 10368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44319 } }
I20260808 23:45:16.666852 10368 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:16.667003 10368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:16.667212 10368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:16.670418 10368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44319 } }
I20260808 23:45:16.671062 10368 leader_election.cc:302] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cee43e9b8dd4ffd9f23622d3a6b6253; no voters:
I20260808 23:45:16.671737 10368 leader_election.cc:288] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:16.671826 10378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:16.673189 10378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [term 1 LEADER]: Becoming Leader. State: Replica: 9cee43e9b8dd4ffd9f23622d3a6b6253, State: Running, Role: LEADER
I20260808 23:45:16.673836 10378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44319 } }
I20260808 23:45:16.674180 10368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:16.676501 10380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cee43e9b8dd4ffd9f23622d3a6b6253. Latest consensus state: current_term: 1 leader_uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44319 } } }
I20260808 23:45:16.676878 10380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:16.677443 10379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cee43e9b8dd4ffd9f23622d3a6b6253" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44319 } } }
I20260808 23:45:16.677994 10379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:16.688313 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:16.666433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53137 (local address 127.18.186.190:44319)
0808 23:45:16.666662 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:16.666671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:16.667096 (+ 425us) server_negotiation.cc:407] Connection header received
0808 23:45:16.667274 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:16.667278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:16.667369 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:16.667541 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:16.669486 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:16.670266 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:16.672121 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:16.672805 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:16.672901 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:16.676870 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:16.676880 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:16.676889 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:16.676945 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:16.681962 (+ 5017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:16.684568 (+ 2606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:16.684577 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:16.684584 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:16.684854 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:16.685633 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:16.685642 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:16.685648 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:16.686163 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:16.686564 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:16.687514 (+ 950us) server_negotiation.cc:299] Negotiation successful
0808 23:45:16.687780 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:45:16.690953 10383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:16.696933 10383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:16.706777 10383 catalog_manager.cc:1347] Generated new cluster ID: 6280dbb3beb64c008046faf70a4c4719
I20260808 23:45:16.706910 10383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:16.734158 10383 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:16.735415 10383 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:16.749601 10383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cee43e9b8dd4ffd9f23622d3a6b6253: Generated new TSK 0
I20260808 23:45:16.750392 10383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:16.762952 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10297
I20260808 23:45:16.764065 10363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:16.858706 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 10297
W20260808 23:45:16.881413 10277 connection.cc:535] client connection to 127.18.186.190:44319 recv error: Network error: failed to read from TLS socket (remote: 127.18.186.190:44319): Connection reset by peer (error 104)
2026-08-08T23:45:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2195 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:16 dist-test-slave-5k54 krb5kdc[10404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:16 dist-test-slave-5k54 krb5kdc[10404](info): set up 2 sockets
Aug 08 23:45:16 dist-test-slave-5k54 krb5kdc[10404](info): commencing operation
krb5kdc: starting...
W20260808 23:45:18.026947 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:18 dist-test-slave-5k54 krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:45:18.347510 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:46765
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:18.684257 10417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:18.684571 10417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:18.684752 10417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:18.692310 10417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:18.692432 10417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:18.692515 10417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:18.692636 10417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:18.692700 10417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:18.702893 10417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:46765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:18.704326 10417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:18.705665 10417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:18.717648 10422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:18.717621 10425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:18.717628 10423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:18.719120 10417 server_base.cc:1034] running on GCE node
I20260808 23:45:18.720129 10417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:18.721486 10417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:18.722719 10417 hybrid_clock.cc:648] HybridClock initialized: now 1786232718722675 us; error 58 us; skew 500 ppm
I20260808 23:45:18.723064 10417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:18 dist-test-slave-5k54 krb5kdc[10404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232718, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:18.726361 10417 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:45:18.727892 10417 webserver.cc:460] Webserver started at http://127.18.186.190:44003/ using document root <none> and password file <none>
I20260808 23:45:18.728561 10417 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:18.728710 10417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:18.729034 10417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:18.732046 10417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "3f11931c49194c55b2cdbece47a3b877"
format_stamp: "Formatted at 2026-08-08 23:45:18 on dist-test-slave-5k54"
server_key: "9d6789479ea0f7754a55c987eecc79ae"
server_key_iv: "43753c62f2a021b5b6bf64f8bbd9d577"
server_key_version: "encryptionkey@0"
I20260808 23:45:18.732796 10417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "3f11931c49194c55b2cdbece47a3b877"
format_stamp: "Formatted at 2026-08-08 23:45:18 on dist-test-slave-5k54"
server_key: "9d6789479ea0f7754a55c987eecc79ae"
server_key_iv: "43753c62f2a021b5b6bf64f8bbd9d577"
server_key_version: "encryptionkey@0"
I20260808 23:45:18.738029 10417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:45:18.742349 10433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:18.743754 10417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:45:18.743968 10417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "3f11931c49194c55b2cdbece47a3b877"
format_stamp: "Formatted at 2026-08-08 23:45:18 on dist-test-slave-5k54"
server_key: "9d6789479ea0f7754a55c987eecc79ae"
server_key_iv: "43753c62f2a021b5b6bf64f8bbd9d577"
server_key_version: "encryptionkey@0"
I20260808 23:45:18.744143 10417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:18.765220 10417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:18.769769 10417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:18.770099 10417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:18.789577 10417 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:46765
I20260808 23:45:18.789608 10490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:46765 every 8 connection(s)
I20260808 23:45:18.790921 10417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:18.794675 10491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:18.795010 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 10417
I20260808 23:45:18.795199 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:18.795770 19178 external_mini_cluster.cc:1428] Setting key b74da36db48add5f607fe3adc4e65384
I20260808 23:45:18.809162 10491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877: Bootstrap starting.
I20260808 23:45:18.811807 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:18.797631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35359 (local address 127.18.186.190:46765)
0808 23:45:18.798422 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:18.798437 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:18.798488 (+ 51us) server_negotiation.cc:407] Connection header received
0808 23:45:18.801446 (+ 2958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:18.801473 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:18.801993 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:18.802420 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:18.802926 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:18.804837 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:18.806242 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:18.807317 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:18.807514 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:18.809010 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:18.809020 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:18.809037 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:18.809720 (+ 683us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:18.809818 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:18.809849 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:18.809997 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:18.810270 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 23:45:18.810433 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":333,"threads_started":1}
I20260808 23:45:18.812933 10491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:18.814013 10491 log.cc:824] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:18.815836 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:18.817221 10491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877: No bootstrap required, opened a new log
I20260808 23:45:18.826478 10491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f11931c49194c55b2cdbece47a3b877" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46765 } }
I20260808 23:45:18.826942 10491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:18.827117 10491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f11931c49194c55b2cdbece47a3b877, State: Initialized, Role: FOLLOWER
I20260808 23:45:18.827696 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:18.820483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54353 (local address 127.18.186.190:46765)
0808 23:45:18.820725 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:18.820733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:18.821131 (+ 398us) server_negotiation.cc:407] Connection header received
0808 23:45:18.821301 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:18.821306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:18.821372 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:18.821469 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:18.822071 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:18.822879 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:18.824850 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:18.825392 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:18.825525 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:18.826777 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:18.826782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:18.826786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:18.826807 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:18.826864 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:18.826884 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:18.827075 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:18.827278 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 23:45:18.827334 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:45:18.827915 10491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f11931c49194c55b2cdbece47a3b877" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46765 } }
I20260808 23:45:18.828238 10491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:18.828408 10491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:18.828603 10491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:18.831151 10491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f11931c49194c55b2cdbece47a3b877" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46765 } }
I20260808 23:45:18.831728 10491 leader_election.cc:302] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f11931c49194c55b2cdbece47a3b877; no voters:
I20260808 23:45:18.832314 10491 leader_election.cc:288] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:18.832453 10502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:18.833244 10502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [term 1 LEADER]: Becoming Leader. State: Replica: 3f11931c49194c55b2cdbece47a3b877, State: Running, Role: LEADER
I20260808 23:45:18.833843 10502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f11931c49194c55b2cdbece47a3b877" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46765 } }
I20260808 23:45:18.834537 10491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:18.837047 10503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f11931c49194c55b2cdbece47a3b877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f11931c49194c55b2cdbece47a3b877" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46765 } } }
I20260808 23:45:18.837700 10503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:18.836891 10504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f11931c49194c55b2cdbece47a3b877. Latest consensus state: current_term: 1 leader_uuid: "3f11931c49194c55b2cdbece47a3b877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f11931c49194c55b2cdbece47a3b877" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 46765 } } }
I20260808 23:45:18.838706 10504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:18.839270 10509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:18.844447 10509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:18.854071 10509 catalog_manager.cc:1347] Generated new cluster ID: 7fdf90464e064d26be9db68dad404c26
I20260808 23:45:18.854185 10509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:18.874188 10509 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:18.876179 10509 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:18.889219 10509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f11931c49194c55b2cdbece47a3b877: Generated new TSK 0
I20260808 23:45:18.890425 10509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:18.903541 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10417
W20260808 23:45:18.904681 10486 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35359
W20260808 23:45:18.905452 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10417: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:18.905614 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 10417
W20260808 23:45:18.926901 10402 connection.cc:535] client connection to 127.18.186.190:46765 recv error: Network error: failed to read from TLS socket (remote: 127.18.186.190:46765): Connection reset by peer (error 104)
2026-08-08T23:45:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2046 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:19 dist-test-slave-5k54 krb5kdc[10523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:19 dist-test-slave-5k54 krb5kdc[10523](info): set up 2 sockets
Aug 08 23:45:19 dist-test-slave-5k54 krb5kdc[10523](info): commencing operation
krb5kdc: starting...
W20260808 23:45:20.079777 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:20 dist-test-slave-5k54 krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:45:20.387938 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44871
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:35545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:44871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:20.731606 10537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:20.731884 10537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:20.731982 10537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:20.739079 10537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:20.739181 10537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:20.739229 10537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:20.739290 10537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:20.739334 10537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:20.750659 10537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:35545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:44871
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:44871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:20.752014 10537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:20.753309 10537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:20.765175 10542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:20.765268 10545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:20.765236 10543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:20.766033 10537 server_base.cc:1034] running on GCE node
I20260808 23:45:20.766800 10537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:20.768258 10537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:20.769527 10537 hybrid_clock.cc:648] HybridClock initialized: now 1786232720769477 us; error 39 us; skew 500 ppm
I20260808 23:45:20.769960 10537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:20 dist-test-slave-5k54 krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:20.773289 10537 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:45:20.774740 10537 webserver.cc:460] Webserver started at http://127.18.186.190:43085/ using document root <none> and password file <none>
I20260808 23:45:20.775478 10537 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:20.775641 10537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:20.775966 10537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:20.779013 10537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "e97aa7df58a34c9f82bec8cb32760651"
format_stamp: "Formatted at 2026-08-08 23:45:20 on dist-test-slave-5k54"
server_key: "fa84b724a438ddbbcffb594fd9c0f977"
server_key_iv: "af810dc496a8e27c1dcf18b02163afe2"
server_key_version: "encryptionkey@0"
I20260808 23:45:20.779836 10537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "e97aa7df58a34c9f82bec8cb32760651"
format_stamp: "Formatted at 2026-08-08 23:45:20 on dist-test-slave-5k54"
server_key: "fa84b724a438ddbbcffb594fd9c0f977"
server_key_iv: "af810dc496a8e27c1dcf18b02163afe2"
server_key_version: "encryptionkey@0"
I20260808 23:45:20.785069 10537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 23:45:20.788753 10552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:20.790185 10537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:45:20.790378 10537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "e97aa7df58a34c9f82bec8cb32760651"
format_stamp: "Formatted at 2026-08-08 23:45:20 on dist-test-slave-5k54"
server_key: "fa84b724a438ddbbcffb594fd9c0f977"
server_key_iv: "af810dc496a8e27c1dcf18b02163afe2"
server_key_version: "encryptionkey@0"
I20260808 23:45:20.790581 10537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:20.796218 10537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:20.800166 10537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:20.800462 10537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:20.819602 10537 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:44871
I20260808 23:45:20.819687 10608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:44871 every 8 connection(s)
I20260808 23:45:20.821132 10537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:20.825050 10609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:20.830200 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 10537
I20260808 23:45:20.830410 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:20.830885 19178 external_mini_cluster.cc:1428] Setting key d0ae9d0e8e12f791e5d17365f3ead35d
I20260808 23:45:20.836277 10609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651: Bootstrap starting.
I20260808 23:45:20.838738 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:20.832794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33411 (local address 127.18.186.190:44871)
0808 23:45:20.833698 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:20.833713 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:20.833755 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:45:20.835415 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:20.835494 (+ 79us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:20.835823 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:20.836139 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:20.837205 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:20.837211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:20.837223 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:20.837237 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:20.837293 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:20.837312 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:20.837495 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:20.837618 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 23:45:20.837819 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":403,"threads_started":1}
I20260808 23:45:20.839884 10609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:20.841012 10609 log.cc:824] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:20.842516 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:20.843956 10609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651: No bootstrap required, opened a new log
I20260808 23:45:20.849545 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:20.847042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33523 (local address 127.18.186.190:44871)
0808 23:45:20.847257 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:20.847266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:20.847661 (+ 395us) server_negotiation.cc:407] Connection header received
0808 23:45:20.847822 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:20.847831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:20.847893 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:20.847964 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:20.848897 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:20.848901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:20.848905 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:20.848918 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:20.848954 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:20.848968 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:20.849053 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:20.849198 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 23:45:20.849244 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 23:45:20.852840 10609 raft_consensus.cc:348] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97aa7df58a34c9f82bec8cb32760651" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44871 } }
I20260808 23:45:20.853199 10609 raft_consensus.cc:374] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:20.853307 10609 raft_consensus.cc:729] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97aa7df58a34c9f82bec8cb32760651, State: Initialized, Role: FOLLOWER
I20260808 23:45:20.853895 10609 consensus_queue.cc:260] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97aa7df58a34c9f82bec8cb32760651" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44871 } }
I20260808 23:45:20.854159 10609 raft_consensus.cc:388] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:20.854319 10609 raft_consensus.cc:482] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:20.854504 10609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:20.857029 10609 raft_consensus.cc:504] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97aa7df58a34c9f82bec8cb32760651" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44871 } }
I20260808 23:45:20.857538 10609 leader_election.cc:302] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e97aa7df58a34c9f82bec8cb32760651; no voters:
I20260808 23:45:20.858122 10609 leader_election.cc:288] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:20.858280 10620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:20.859050 10620 raft_consensus.cc:686] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [term 1 LEADER]: Becoming Leader. State: Replica: e97aa7df58a34c9f82bec8cb32760651, State: Running, Role: LEADER
I20260808 23:45:20.859692 10620 consensus_queue.cc:237] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97aa7df58a34c9f82bec8cb32760651" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44871 } }
I20260808 23:45:20.860551 10609 sys_catalog.cc:564] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:20.862573 10621 sys_catalog.cc:455] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e97aa7df58a34c9f82bec8cb32760651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97aa7df58a34c9f82bec8cb32760651" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44871 } } }
I20260808 23:45:20.862680 10622 sys_catalog.cc:455] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e97aa7df58a34c9f82bec8cb32760651. Latest consensus state: current_term: 1 leader_uuid: "e97aa7df58a34c9f82bec8cb32760651" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97aa7df58a34c9f82bec8cb32760651" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 44871 } } }
I20260808 23:45:20.862972 10621 sys_catalog.cc:458] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:20.863013 10622 sys_catalog.cc:458] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:20.863890 10626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:20.870082 10626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:20.891184 10626 catalog_manager.cc:1347] Generated new cluster ID: 1e0012eceb194ecaa366ac2301e9a4ab
I20260808 23:45:20.891311 10626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:20.910439 10626 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:20.910909 10626 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:20.922868 10626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e97aa7df58a34c9f82bec8cb32760651: Generated new TSK 0
I20260808 23:45:20.923928 10626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:20.941615 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10537
W20260808 23:45:20.942746 10603 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33411
W20260808 23:45:20.943470 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10537: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:20.943614 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 10537
2026-08-08T23:45:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2037 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:21 dist-test-slave-5k54 krb5kdc[10642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:21 dist-test-slave-5k54 krb5kdc[10642](info): set up 2 sockets
Aug 08 23:45:21 dist-test-slave-5k54 krb5kdc[10642](info): commencing operation
krb5kdc: starting...
W20260808 23:45:22.106971 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:22 dist-test-slave-5k54 krb5kdc[10642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:45:22.422981 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42287
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:45087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:22.768180 10655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:22.768452 10655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:22.768637 10655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:22.776139 10655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:22.776274 10655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:22.776384 10655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:22.776494 10655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:22.776571 10655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:22.786403 10655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:45087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:22.787778 10655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:22.789173 10655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:22.801913 10661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:22.802007 10660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:22.801977 10664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:22.803007 10655 server_base.cc:1034] running on GCE node
I20260808 23:45:22.803894 10655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:22.805352 10655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:22.806587 10655 hybrid_clock.cc:648] HybridClock initialized: now 1786232722806542 us; error 42 us; skew 500 ppm
I20260808 23:45:22.806982 10655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:22 dist-test-slave-5k54 krb5kdc[10642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232722, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:22.810247 10655 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:45:22.811913 10655 webserver.cc:460] Webserver started at http://127.18.186.190:34953/ using document root <none> and password file <none>
I20260808 23:45:22.812637 10655 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:22.812812 10655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:22.813140 10655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:22.816344 10655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "06e2a5e70f8f4c6392bd7633cae0238a"
format_stamp: "Formatted at 2026-08-08 23:45:22 on dist-test-slave-5k54"
server_key: "6371a61af509931d9990e9eeaabbb6be"
server_key_iv: "1354af1b80d0de1d4e761af2f5fd89ed"
server_key_version: "encryptionkey@0"
I20260808 23:45:22.817126 10655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "06e2a5e70f8f4c6392bd7633cae0238a"
format_stamp: "Formatted at 2026-08-08 23:45:22 on dist-test-slave-5k54"
server_key: "6371a61af509931d9990e9eeaabbb6be"
server_key_iv: "1354af1b80d0de1d4e761af2f5fd89ed"
server_key_version: "encryptionkey@0"
I20260808 23:45:22.822458 10655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:45:22.826193 10671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:22.827647 10655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:45:22.827881 10655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "06e2a5e70f8f4c6392bd7633cae0238a"
format_stamp: "Formatted at 2026-08-08 23:45:22 on dist-test-slave-5k54"
server_key: "6371a61af509931d9990e9eeaabbb6be"
server_key_iv: "1354af1b80d0de1d4e761af2f5fd89ed"
server_key_version: "encryptionkey@0"
I20260808 23:45:22.828095 10655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:22.848496 10655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:22.852591 10655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:22.852892 10655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:22.872400 10655 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42287
I20260808 23:45:22.872401 10723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42287 every 8 connection(s)
I20260808 23:45:22.874418 10655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:22.879722 10724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:22.883340 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 10655
I20260808 23:45:22.883584 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:22.884088 19178 external_mini_cluster.cc:1428] Setting key 495b8c30df23b937b3bac3c480919c94
I20260808 23:45:22.893416 10724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a: Bootstrap starting.
I20260808 23:45:22.896924 10724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:22.897944 10724 log.cc:824] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:45:22 dist-test-slave-5k54 krb5kdc[10642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:22.901175 10724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a: No bootstrap required, opened a new log
I20260808 23:45:22.909073 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:22.886045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54193 (local address 127.18.186.190:42287)
0808 23:45:22.886889 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:22.886906 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:22.886957 (+ 51us) server_negotiation.cc:407] Connection header received
0808 23:45:22.889186 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:22.889216 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:22.889776 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:22.890212 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:22.891667 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:22.893666 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:22.894968 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:22.896019 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:22.896231 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:22.900491 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:22.900501 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:22.900520 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:22.900573 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:22.904993 (+ 4420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:22.905675 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:22.905689 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:22.905694 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:22.905873 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:22.906256 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:22.906261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:22.906264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:22.906796 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:22.907175 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:22.907630 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 23:45:22.908045 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":366,"threads_started":1}
I20260808 23:45:22.910742 10724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e2a5e70f8f4c6392bd7633cae0238a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42287 } }
I20260808 23:45:22.911131 10724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:22.911247 10724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e2a5e70f8f4c6392bd7633cae0238a, State: Initialized, Role: FOLLOWER
I20260808 23:45:22.911907 10724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e2a5e70f8f4c6392bd7633cae0238a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42287 } }
I20260808 23:45:22.912206 10724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:22.912428 10724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:22.912609 10724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:22.912940 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:22.915908 10724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e2a5e70f8f4c6392bd7633cae0238a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42287 } }
I20260808 23:45:22.916538 10724 leader_election.cc:302] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06e2a5e70f8f4c6392bd7633cae0238a; no voters:
I20260808 23:45:22.917307 10724 leader_election.cc:288] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:22.917415 10734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:22.918303 10734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [term 1 LEADER]: Becoming Leader. State: Replica: 06e2a5e70f8f4c6392bd7633cae0238a, State: Running, Role: LEADER
I20260808 23:45:22.919070 10734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e2a5e70f8f4c6392bd7633cae0238a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42287 } }
I20260808 23:45:22.919732 10724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:22.922703 10737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06e2a5e70f8f4c6392bd7633cae0238a. Latest consensus state: current_term: 1 leader_uuid: "06e2a5e70f8f4c6392bd7633cae0238a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e2a5e70f8f4c6392bd7633cae0238a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42287 } } }
I20260808 23:45:22.923193 10737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:22.933951 10743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:22.933818 10736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06e2a5e70f8f4c6392bd7633cae0238a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e2a5e70f8f4c6392bd7633cae0238a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42287 } } }
I20260808 23:45:22.934199 10736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:22.939702 10743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:22.942978 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:22.918588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43613 (local address 127.18.186.190:42287)
0808 23:45:22.918758 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:22.918775 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:22.922349 (+ 3574us) server_negotiation.cc:407] Connection header received
0808 23:45:22.922798 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:22.922808 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:22.922913 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:22.922986 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:22.925328 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:22.926818 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:22.929241 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:22.930266 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:22.930383 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:22.934113 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:22.934124 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:22.934129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:22.934165 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:22.938459 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:22.939162 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:22.939169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:22.939172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:22.939328 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:22.939732 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:22.939736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:22.939743 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:22.940342 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:22.941927 (+ 1585us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:22.942122 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 23:45:22.942429 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:45:22.959597 10743 catalog_manager.cc:1347] Generated new cluster ID: da2f56d235774fa58d94d1f03ba6cf16
I20260808 23:45:22.959770 10743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:22.967962 10743 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:22.969856 10743 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:22.978350 10743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06e2a5e70f8f4c6392bd7633cae0238a: Generated new TSK 0
I20260808 23:45:22.979410 10743 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:22.988945 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10655
I20260808 23:45:22.990000 10719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:23.081838 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 10655
2026-08-08T23:45:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2136 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:23 dist-test-slave-5k54 krb5kdc[10759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:23 dist-test-slave-5k54 krb5kdc[10759](info): set up 2 sockets
Aug 08 23:45:23 dist-test-slave-5k54 krb5kdc[10759](info): commencing operation
krb5kdc: starting...
W20260808 23:45:24.261646 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:24 dist-test-slave-5k54 krb5kdc[10759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:45:24.592080 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:40485
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:33673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:24.941709 10773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:24.942018 10773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:24.942183 10773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:24.949488 10773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:24.949602 10773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:24.949747 10773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:24.949853 10773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:24.949935 10773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:24.959797 10773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:33673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:40485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:24.961155 10773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:24.962495 10773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:24.974478 10778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:24.974498 10781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:24.974431 10779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:24.975391 10773 server_base.cc:1034] running on GCE node
I20260808 23:45:24.976189 10773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:24.977586 10773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:24.978812 10773 hybrid_clock.cc:648] HybridClock initialized: now 1786232724978746 us; error 52 us; skew 500 ppm
I20260808 23:45:24.979211 10773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:24 dist-test-slave-5k54 krb5kdc[10759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232724, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:24.989041 10773 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:45:24.990443 10773 webserver.cc:460] Webserver started at http://127.18.186.190:45255/ using document root <none> and password file <none>
I20260808 23:45:24.991125 10773 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:24.991232 10773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:24.991564 10773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:24.994549 10773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "6172e5f0210040e8b79339465a61a110"
format_stamp: "Formatted at 2026-08-08 23:45:24 on dist-test-slave-5k54"
server_key: "78d917a83c9813bef34ef0952ccbd7e1"
server_key_iv: "a02750c69f2d7edabe3bdd0e0c0e97fa"
server_key_version: "encryptionkey@0"
I20260808 23:45:24.995240 10773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "6172e5f0210040e8b79339465a61a110"
format_stamp: "Formatted at 2026-08-08 23:45:24 on dist-test-slave-5k54"
server_key: "78d917a83c9813bef34ef0952ccbd7e1"
server_key_iv: "a02750c69f2d7edabe3bdd0e0c0e97fa"
server_key_version: "encryptionkey@0"
I20260808 23:45:25.000825 10773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:45:25.004320 10789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:25.005708 10773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:45:25.005856 10773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "6172e5f0210040e8b79339465a61a110"
format_stamp: "Formatted at 2026-08-08 23:45:24 on dist-test-slave-5k54"
server_key: "78d917a83c9813bef34ef0952ccbd7e1"
server_key_iv: "a02750c69f2d7edabe3bdd0e0c0e97fa"
server_key_version: "encryptionkey@0"
I20260808 23:45:25.006029 10773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:25.063743 10773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:25.068435 10773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:25.068689 10773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:25.087781 10773 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:40485
I20260808 23:45:25.087790 10850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:40485 every 8 connection(s)
I20260808 23:45:25.089215 10773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:25.093111 10851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:25.095220 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 10773
I20260808 23:45:25.095474 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:25.095927 19178 external_mini_cluster.cc:1428] Setting key 52f33d8216b23994d964dabf06e1fdcb
I20260808 23:45:25.105540 10851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110: Bootstrap starting.
I20260808 23:45:25.109658 10851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:25.110581 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:25.097810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46783 (local address 127.18.186.190:40485)
0808 23:45:25.098673 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:25.098687 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:25.098732 (+ 45us) server_negotiation.cc:407] Connection header received
0808 23:45:25.100513 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:25.100537 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:25.100952 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:25.101278 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:25.101972 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:25.103402 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:25.105434 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:25.106254 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:25.106400 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:25.107793 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:25.107801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:25.107816 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:25.108456 (+ 640us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:25.108540 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:25.108568 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:25.108667 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:25.108951 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 23:45:25.109266 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":394,"threads_started":1}
I20260808 23:45:25.111002 10851 log.cc:824] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:25.114349 10851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110: No bootstrap required, opened a new log
I20260808 23:45:25.114823 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:25.121971 10851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6172e5f0210040e8b79339465a61a110" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 40485 } }
I20260808 23:45:25.122373 10851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:25.122503 10851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6172e5f0210040e8b79339465a61a110, State: Initialized, Role: FOLLOWER
I20260808 23:45:25.123229 10851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6172e5f0210040e8b79339465a61a110" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 40485 } }
I20260808 23:45:25.123521 10851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:25.123698 10851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:25.123884 10851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:25.126976 10851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6172e5f0210040e8b79339465a61a110" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 40485 } }
I20260808 23:45:25.127678 10851 leader_election.cc:302] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6172e5f0210040e8b79339465a61a110; no voters:
I20260808 23:45:25.128351 10851 leader_election.cc:288] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:25.128545 10861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:25.130569 10851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:25.131834 10861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [term 1 LEADER]: Becoming Leader. State: Replica: 6172e5f0210040e8b79339465a61a110, State: Running, Role: LEADER
I20260808 23:45:25.132516 10861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6172e5f0210040e8b79339465a61a110" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 40485 } }
I20260808 23:45:25.133373 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:25.124485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53837 (local address 127.18.186.190:40485)
0808 23:45:25.124700 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:25.124706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:25.125304 (+ 598us) server_negotiation.cc:407] Connection header received
0808 23:45:25.125445 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:25.125456 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:25.125538 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:25.125662 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:25.126173 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:25.127236 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:25.129258 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:25.130258 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:25.130386 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:25.132201 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:25.132207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:25.132211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:25.132237 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:25.132289 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:25.132315 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:25.132415 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:25.132944 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 23:45:25.133041 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:45:25.139508 10863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6172e5f0210040e8b79339465a61a110. Latest consensus state: current_term: 1 leader_uuid: "6172e5f0210040e8b79339465a61a110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6172e5f0210040e8b79339465a61a110" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 40485 } } }
I20260808 23:45:25.139920 10863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:25.139460 10862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6172e5f0210040e8b79339465a61a110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6172e5f0210040e8b79339465a61a110" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 40485 } } }
I20260808 23:45:25.140523 10862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:25.142131 10871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:25.147400 10871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:25.156783 10871 catalog_manager.cc:1347] Generated new cluster ID: 1e077e4dbee84b1ab582f6658f0c6998
I20260808 23:45:25.156908 10871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:25.193535 10871 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:25.195587 10871 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:25.204114 10871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6172e5f0210040e8b79339465a61a110: Generated new TSK 0
I20260808 23:45:25.205170 10871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:25.213982 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10773
W20260808 23:45:25.215288 10846 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46783
W20260808 23:45:25.216002 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10773: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:25.216161 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 10773
2026-08-08T23:45:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2133 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:25 dist-test-slave-5k54 krb5kdc[10883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:25 dist-test-slave-5k54 krb5kdc[10883](info): set up 2 sockets
Aug 08 23:45:25 dist-test-slave-5k54 krb5kdc[10883](info): commencing operation
krb5kdc: starting...
W20260808 23:45:26.418620 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:26 dist-test-slave-5k54 krb5kdc[10883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:45:26.751276 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:35831
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:38609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:35831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:27.102653 10897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:27.102949 10897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:27.103116 10897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:27.110498 10897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:27.110613 10897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:27.110728 10897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:27.110836 10897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:27.110922 10897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:27.121007 10897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:38609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:35831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:35831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:27.122470 10897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:27.123900 10897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:27.136034 10902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:27.136018 10903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:27.136236 10905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:27.137058 10897 server_base.cc:1034] running on GCE node
I20260808 23:45:27.137956 10897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:27.139410 10897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:27.140700 10897 hybrid_clock.cc:648] HybridClock initialized: now 1786232727140660 us; error 34 us; skew 500 ppm
I20260808 23:45:27.141072 10897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:27 dist-test-slave-5k54 krb5kdc[10883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232727, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:27.144412 10897 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:45:27.146044 10897 webserver.cc:460] Webserver started at http://127.18.186.190:43237/ using document root <none> and password file <none>
I20260808 23:45:27.146736 10897 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:27.146883 10897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:27.147189 10897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:27.150306 10897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "b76b821190ad4a339140e8eb1673d856"
format_stamp: "Formatted at 2026-08-08 23:45:27 on dist-test-slave-5k54"
server_key: "642bf0e9a05dc82addb0ae1660582a55"
server_key_iv: "575a742099b67c8c72d282a2216dee7a"
server_key_version: "encryptionkey@0"
I20260808 23:45:27.151091 10897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "b76b821190ad4a339140e8eb1673d856"
format_stamp: "Formatted at 2026-08-08 23:45:27 on dist-test-slave-5k54"
server_key: "642bf0e9a05dc82addb0ae1660582a55"
server_key_iv: "575a742099b67c8c72d282a2216dee7a"
server_key_version: "encryptionkey@0"
I20260808 23:45:27.156390 10897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 23:45:27.159915 10914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:27.161180 10897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 23:45:27.161373 10897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "b76b821190ad4a339140e8eb1673d856"
format_stamp: "Formatted at 2026-08-08 23:45:27 on dist-test-slave-5k54"
server_key: "642bf0e9a05dc82addb0ae1660582a55"
server_key_iv: "575a742099b67c8c72d282a2216dee7a"
server_key_version: "encryptionkey@0"
I20260808 23:45:27.161590 10897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:27.167520 10897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:27.171378 10897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:27.171756 10897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:27.190523 10897 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:35831
I20260808 23:45:27.190553 10969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:35831 every 8 connection(s)
I20260808 23:45:27.191946 10897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:27.195374 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 10897
I20260808 23:45:27.195607 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:27.195956 10970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:27.196161 19178 external_mini_cluster.cc:1428] Setting key 4e01dac38a77e200f79a843c4a72007f
I20260808 23:45:27.204957 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:27.197928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39515 (local address 127.18.186.190:35831)
0808 23:45:27.198806 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:27.198822 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:27.198878 (+ 56us) server_negotiation.cc:407] Connection header received
0808 23:45:27.201077 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:27.201105 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:27.201558 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:27.202000 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:27.202987 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:27.202995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:27.203015 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:27.203033 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:27.203103 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:27.203135 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:27.203379 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:27.203534 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 23:45:27.203731 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":358,"threads_started":1}
I20260808 23:45:27.208451 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:27.210350 10970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856: Bootstrap starting.
I20260808 23:45:27.213783 10970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:27.215013 10970 log.cc:824] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:27.215253 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:27.212821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57311 (local address 127.18.186.190:35831)
0808 23:45:27.212992 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:27.212997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:27.213352 (+ 355us) server_negotiation.cc:407] Connection header received
0808 23:45:27.213475 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:27.213479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:27.213543 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:27.213663 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:27.214587 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:27.214592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:27.214597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:45:27.214616 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:27.214671 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:45:27.214690 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:27.214813 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:27.214893 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 23:45:27.214947 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:45:27.218083 10970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856: No bootstrap required, opened a new log
I20260808 23:45:27.226197 10970 raft_consensus.cc:348] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76b821190ad4a339140e8eb1673d856" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 35831 } }
I20260808 23:45:27.226564 10970 raft_consensus.cc:374] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:45:27.226637 10970 raft_consensus.cc:729] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b76b821190ad4a339140e8eb1673d856, State: Initialized, Role: FOLLOWER
I20260808 23:45:27.227130 10970 consensus_queue.cc:260] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76b821190ad4a339140e8eb1673d856" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 35831 } }
I20260808 23:45:27.227331 10970 raft_consensus.cc:388] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:45:27.227475 10970 raft_consensus.cc:482] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:45:27.227622 10970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:27.229709 10970 raft_consensus.cc:504] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76b821190ad4a339140e8eb1673d856" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 35831 } }
I20260808 23:45:27.230146 10970 leader_election.cc:302] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b76b821190ad4a339140e8eb1673d856; no voters:
I20260808 23:45:27.230716 10970 leader_election.cc:288] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:45:27.230914 10980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:27.231635 10980 raft_consensus.cc:686] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [term 1 LEADER]: Becoming Leader. State: Replica: b76b821190ad4a339140e8eb1673d856, State: Running, Role: LEADER
I20260808 23:45:27.232491 10980 consensus_queue.cc:237] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76b821190ad4a339140e8eb1673d856" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 35831 } }
I20260808 23:45:27.233014 10970 sys_catalog.cc:564] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:27.235742 10982 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b76b821190ad4a339140e8eb1673d856. Latest consensus state: current_term: 1 leader_uuid: "b76b821190ad4a339140e8eb1673d856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76b821190ad4a339140e8eb1673d856" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 35831 } } }
I20260808 23:45:27.236214 10982 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:27.236665 10981 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b76b821190ad4a339140e8eb1673d856" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76b821190ad4a339140e8eb1673d856" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 35831 } } }
I20260808 23:45:27.236953 10981 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:27.240163 10989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:27.245335 10989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:45:27.253296 10989 catalog_manager.cc:1347] Generated new cluster ID: 94d7b265fefc445994ed7651dbf98375
I20260808 23:45:27.253408 10989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:27.266995 10989 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:27.267475 10989 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:27.283795 10989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b76b821190ad4a339140e8eb1673d856: Generated new TSK 0
I20260808 23:45:27.284623 10989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:27.296150 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10897
W20260808 23:45:27.297463 10965 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39515
W20260808 23:45:27.298286 19178 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 10897: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:45:27.298475 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 10897
2026-08-08T23:45:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2085 ms)
[----------] 18 tests from AuthTokenIssuingTest (37457 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:27 dist-test-slave-5k54 krb5kdc[11002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:27 dist-test-slave-5k54 krb5kdc[11002](info): set up 2 sockets
Aug 08 23:45:27 dist-test-slave-5k54 krb5kdc[11002](info): commencing operation
krb5kdc: starting...
W20260808 23:45:28.490464 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:28 dist-test-slave-5k54 krb5kdc[11002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:45:28.813330 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:43921
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:43921,127.18.186.189:46167,127.18.186.188:33435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:29.157148 11015 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:45:29.157510 11015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:29.157629 11015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:29.157773 11015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:29.165419 11015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:29.165514 11015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:29.165617 11015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:29.165707 11015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:29.165827 11015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:29.175647 11015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:43921,127.18.186.189:46167,127.18.186.188:33435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:43921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:29.176956 11015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:29.178305 11015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:29.190557 11020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:29.190474 11023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:29.190514 11021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:29.191362 11015 server_base.cc:1034] running on GCE node
I20260808 23:45:29.192234 11015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:29.193607 11015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:29.194907 11015 hybrid_clock.cc:648] HybridClock initialized: now 1786232729194844 us; error 31 us; skew 500 ppm
I20260808 23:45:29.195252 11015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:29 dist-test-slave-5k54 krb5kdc[11002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:29.198531 11015 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:45:29.200103 11015 webserver.cc:460] Webserver started at http://127.18.186.190:45473/ using document root <none> and password file <none>
I20260808 23:45:29.200799 11015 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:29.200932 11015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:29.201262 11015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:29.204291 11015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "9a45097c1a6843889e78106749f7a96a"
format_stamp: "Formatted at 2026-08-08 23:45:29 on dist-test-slave-5k54"
server_key: "4cf520ebbf58e730831a0c60cbf39de2"
server_key_iv: "baea60ed4e0c1414747130ac78cf771f"
server_key_version: "encryptionkey@0"
I20260808 23:45:29.205125 11015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "9a45097c1a6843889e78106749f7a96a"
format_stamp: "Formatted at 2026-08-08 23:45:29 on dist-test-slave-5k54"
server_key: "4cf520ebbf58e730831a0c60cbf39de2"
server_key_iv: "baea60ed4e0c1414747130ac78cf771f"
server_key_version: "encryptionkey@0"
I20260808 23:45:29.210225 11015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:45:29.213778 11031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:29.215329 11015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 23:45:29.215579 11015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "9a45097c1a6843889e78106749f7a96a"
format_stamp: "Formatted at 2026-08-08 23:45:29 on dist-test-slave-5k54"
server_key: "4cf520ebbf58e730831a0c60cbf39de2"
server_key_iv: "baea60ed4e0c1414747130ac78cf771f"
server_key_version: "encryptionkey@0"
I20260808 23:45:29.215816 11015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:29.233283 11015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:29.238657 11015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:29.238963 11015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:29.257882 11015 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:43921
I20260808 23:45:29.257911 11083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:43921 every 8 connection(s)
I20260808 23:45:29.259297 11015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:29.263128 11084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:29.265292 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 11015
I20260808 23:45:29.265544 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:29.266218 19178 external_mini_cluster.cc:1428] Setting key 66df0ac19572cd1aa930264ae1d9b7c8
I20260808 23:45:29.269938 11084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:45:29 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:29.292748 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.273166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.18.186.190:43921)
0808 23:45:29.274873 (+ 1707us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:29.274881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:29.274930 (+ 49us) server_negotiation.cc:407] Connection header received
0808 23:45:29.276522 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:29.276528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:29.276745 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:29.276966 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:29.278247 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.279500 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.280895 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.281451 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.281658 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:29.285653 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:29.285661 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:29.285671 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:29.285700 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:29.288526 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:29.289453 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:29.289459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:29.289461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:29.289606 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:29.290064 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:29.290067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:29.290069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:29.290533 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:29.290690 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:29.291339 (+ 649us) server_negotiation.cc:299] Negotiation successful
0808 23:45:29.291668 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1480,"thread_start_us":1261,"threads_started":1}
I20260808 23:45:29.292810 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.273165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43921 (local address 127.0.0.1:48424)
0808 23:45:29.274042 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.274087 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:29.276158 (+ 2071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:29.276982 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:29.276988 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:29.277705 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:29.278034 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:29.278051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.279654 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.279660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:29.280745 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:29.280756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.281677 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.281684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:29.281866 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:29.282558 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:29.282579 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:29.285408 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:29.288665 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:29.288680 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:29.288694 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:29.289308 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:29.289725 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:29.289732 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:29.289738 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:29.289957 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:29.290688 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:29.290699 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:29.290960 (+ 261us) client_negotiation.cc:769] Sending connection context
0808 23:45:29.291248 (+ 288us) client_negotiation.cc:240] Negotiation successful
0808 23:45:29.291833 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":51,"thread_start_us":390,"threads_started":1}
I20260808 23:45:29.295871 11084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:45:29.296938 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.296564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42676)
0808 23:45:29.296724 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.296825 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 23:45:29.297163 11033 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
I20260808 23:45:29.298787 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.298431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42692)
0808 23:45:29.298603 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.298672 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 23:45:29.299001 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46167: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.186.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab.
Entry for principal kudu/127.18.186.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab.
I20260808 23:45:29.338995 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.189:46167
--webserver_interface=127.18.186.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab
--principal=kudu/127.18.186.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:43921,127.18.186.189:46167,127.18.186.188:33435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:45:29.348335 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } attempt: 1
I20260808 23:45:29.349740 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.349134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42708)
0808 23:45:29.349389 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.349544 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 23:45:29.350925 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.350511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42724)
0808 23:45:29.350671 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.350755 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:45:29.351197 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46167: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
I20260808 23:45:29.419481 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } attempt: 2
I20260808 23:45:29.420858 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.420346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42740)
0808 23:45:29.420609 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.420701 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 23:45:29.421952 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.421575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42744)
0808 23:45:29.421739 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.421804 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:45:29.422178 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46167: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
I20260808 23:45:29.513496 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } attempt: 3
I20260808 23:45:29.514786 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.514325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42748)
0808 23:45:29.514545 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.514646 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 23:45:29.515846 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.515514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42752)
0808 23:45:29.515641 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.515711 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 23:45:29.516101 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46167: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
I20260808 23:45:29.659377 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } attempt: 4
I20260808 23:45:29.660714 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.660156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42762)
0808 23:45:29.660481 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.660570 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260808 23:45:29.661738 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.661395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42768)
0808 23:45:29.661537 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.661622 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:45:29.662001 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:46167: Network error: Client connection negotiation failed: client connection to 127.18.186.189:46167: connect: Connection refused (error 111)
W20260808 23:45:29.680323 11092 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:45:29.680703 11092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:29.680815 11092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:29.680944 11092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:29.688359 11092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:29.688462 11092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:29.688567 11092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:29.688663 11092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:29.688777 11092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:29.699167 11092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:43921,127.18.186.189:46167,127.18.186.188:33435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab
--principal=kudu/127.18.186.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.189:46167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.186.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:29.700678 11092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:29.701992 11092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:29.713923 11097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:29.713950 11098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:29.714645 11100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:29.714723 11092 server_base.cc:1034] running on GCE node
I20260808 23:45:29.715651 11092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:29.717103 11092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:29.718315 11092 hybrid_clock.cc:648] HybridClock initialized: now 1786232729718245 us; error 49 us; skew 500 ppm
I20260808 23:45:29.718650 11092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:29 dist-test-slave-5k54 krb5kdc[11002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:29.722146 11092 init.cc:377] Logged in from keytab as kudu/127.18.186.189@KRBTEST.COM (short username kudu)
I20260808 23:45:29.723609 11092 webserver.cc:460] Webserver started at http://127.18.186.189:46153/ using document root <none> and password file <none>
I20260808 23:45:29.724294 11092 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:29.724412 11092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:29.724741 11092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:29.727780 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/data/instance:
uuid: "5c667a7edc9f42a7a607efc8cfd7ac02"
format_stamp: "Formatted at 2026-08-08 23:45:29 on dist-test-slave-5k54"
server_key: "af7daf11aab62b2304c7d3ac019d375a"
server_key_iv: "5f9143a94174e9d5f8ebc951560d8deb"
server_key_version: "encryptionkey@0"
I20260808 23:45:29.728564 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/wal/instance:
uuid: "5c667a7edc9f42a7a607efc8cfd7ac02"
format_stamp: "Formatted at 2026-08-08 23:45:29 on dist-test-slave-5k54"
server_key: "af7daf11aab62b2304c7d3ac019d375a"
server_key_iv: "5f9143a94174e9d5f8ebc951560d8deb"
server_key_version: "encryptionkey@0"
I20260808 23:45:29.733680 11092 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260808 23:45:29.737192 11107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:29.738612 11092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:45:29.738791 11092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/wal
uuid: "5c667a7edc9f42a7a607efc8cfd7ac02"
format_stamp: "Formatted at 2026-08-08 23:45:29 on dist-test-slave-5k54"
server_key: "af7daf11aab62b2304c7d3ac019d375a"
server_key_iv: "5f9143a94174e9d5f8ebc951560d8deb"
server_key_version: "encryptionkey@0"
I20260808 23:45:29.739023 11092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:29.761473 11092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:29.765479 11092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:29.765764 11092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:29.787143 11092 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.189:46167
I20260808 23:45:29.787164 11166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.189:46167 every 8 connection(s)
I20260808 23:45:29.788575 11092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
I20260808 23:45:29.791966 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 11092
I20260808 23:45:29.792173 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-1/wal/instance
I20260808 23:45:29.792474 11167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:29.792682 19178 external_mini_cluster.cc:1428] Setting key 8557853b809c01092eedf9862bb71d70
I20260808 23:45:29.799343 11167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:45:29 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:29.823225 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.802162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.18.186.190:43921)
0808 23:45:29.802687 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:29.802692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:29.803823 (+ 1131us) server_negotiation.cc:407] Connection header received
0808 23:45:29.805826 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:29.805830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:29.805902 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:29.805997 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:29.807982 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.809282 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.810795 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.811552 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.811705 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:29.815229 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:29.815238 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:29.815245 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:29.815299 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:29.819214 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:29.820249 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:29.820257 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:29.820264 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:29.820444 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:29.820890 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:29.820898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:29.820904 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:29.821332 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:29.821504 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:29.822407 (+ 903us) server_negotiation.cc:299] Negotiation successful
0808 23:45:29.822690 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":301,"threads_started":1}
I20260808 23:45:29.823853 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.802830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43921 (local address 127.0.0.1:48426)
0808 23:45:29.803662 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.803700 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:29.805503 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:29.806285 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:29.806298 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:29.807121 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:29.807768 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:29.807785 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.809450 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.809454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:29.810644 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:29.810656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.811772 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.811777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:29.811994 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:29.812693 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:29.812720 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:29.814967 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:29.819463 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:29.819474 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:29.819496 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:29.820113 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:29.820572 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:29.820576 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:29.820578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:29.820761 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:29.821524 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:29.821535 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:29.821988 (+ 453us) client_negotiation.cc:769] Sending connection context
0808 23:45:29.822221 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 23:45:29.822616 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":324,"threads_started":1}
I20260808 23:45:29.825764 11167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.186.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.188@KRBTEST.COM" created.
Aug 08 23:45:29 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:45:29.841328 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.826881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.18.186.189:46167)
0808 23:45:29.827581 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:29.827587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:29.827613 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:45:29.827657 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:29.827662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:29.827837 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:29.827990 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:29.828900 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.830025 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.831258 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.831836 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.831990 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:29.835522 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:29.835542 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:29.835556 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:29.835592 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:29.838487 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:29.839181 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:29.839196 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:29.839198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:29.839318 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:29.839719 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:29.839727 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:29.839730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:29.840085 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:29.840309 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:29.840671 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 23:45:29.840930 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":496,"thread_start_us":353,"threads_started":1}
I20260808 23:45:29.841361 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.826557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42774)
0808 23:45:29.826748 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.826771 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:29.826882 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:29.828017 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:29.828021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:29.828465 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:29.828685 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:29.828698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.830163 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.830170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:29.831091 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:29.831115 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.831998 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.832003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:29.832205 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:29.832868 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:29.832885 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:29.835229 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:29.838641 (+ 3412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:29.838648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:29.838654 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:29.839044 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:29.839457 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:29.839462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:29.839465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:29.839613 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:29.840224 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:29.840229 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:29.840383 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 23:45:29.840534 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 23:45:29.840815 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 23:45:29.844120 11167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:45:29.845379 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.844894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50324)
0808 23:45:29.845088 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.845212 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260808 23:45:29.845679 11109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
I20260808 23:45:29.847283 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.846923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50338)
0808 23:45:29.847106 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.847180 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 23:45:29.847532 11167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab.
Entry for principal kudu/127.18.186.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab.
I20260808 23:45:29.866884 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.188:33435
--webserver_interface=127.18.186.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab
--principal=kudu/127.18.186.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:43921,127.18.186.189:46167,127.18.186.188:33435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:45:29.896852 11167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 1
I20260808 23:45:29.898152 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.897616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50346)
0808 23:45:29.897820 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.897974 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 23:45:29.899205 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.898846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50356)
0808 23:45:29.899027 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.899100 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 23:45:29.899467 11167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
I20260808 23:45:29.961283 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } attempt: 5
I20260808 23:45:29.967700 11167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 2
I20260808 23:45:29.968806 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.968246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50366)
0808 23:45:29.968460 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.968579 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 23:45:29.969871 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.969530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50368)
0808 23:45:29.969676 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.969750 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
Aug 08 23:45:29 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
W20260808 23:45:29.970166 11167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
I20260808 23:45:29.976162 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.962194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.18.186.189:46167)
0808 23:45:29.962387 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:29.962391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:29.962411 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:45:29.962617 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:29.962621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:29.962685 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:29.962772 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:29.964023 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.964736 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.966396 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.966849 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.966953 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:29.970906 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:29.970911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:29.970914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:29.970944 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:29.973625 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:29.974404 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:29.974408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:29.974410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:29.974526 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:29.974930 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:29.974933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:29.974935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:29.975194 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:29.975303 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:29.975789 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 23:45:29.975953 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:45:29.976368 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.962076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42782)
0808 23:45:29.962301 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.962324 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:29.962469 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:29.962857 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:29.962862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:29.963563 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:29.963841 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:29.963859 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.964887 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.964893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:29.966231 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:29.966246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:29.967004 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:29.967010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:29.967240 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:29.968077 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:29.968108 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:29.970727 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:29.973758 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:29.973763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:29.973767 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:29.974279 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:29.974633 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:29.974638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:29.974641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:29.974826 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:29.975310 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:29.975315 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:29.975521 (+ 206us) client_negotiation.cc:769] Sending connection context
0808 23:45:29.975702 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 23:45:29.976028 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 23:45:29.977519 11084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:45:29.978677 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.978224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50372)
0808 23:45:29.978419 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.978516 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 23:45:29.979652 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:29.979280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50386)
0808 23:45:29.979461 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:29.979544 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 23:45:29.979878 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
I20260808 23:45:30.031147 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 1
I20260808 23:45:30.032542 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.031969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50392)
0808 23:45:30.032198 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.032313 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 23:45:30.033620 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.033236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50408)
0808 23:45:30.033399 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.033491 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 23:45:30.033838 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
I20260808 23:45:30.061422 11167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 3
I20260808 23:45:30.062810 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.062117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50410)
0808 23:45:30.062414 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.062557 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 23:45:30.064157 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.063649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50426)
0808 23:45:30.063907 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.064005 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260808 23:45:30.064465 11167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
I20260808 23:45:30.102077 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 2
I20260808 23:45:30.103564 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.102809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50430)
0808 23:45:30.103125 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.103261 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260808 23:45:30.104662 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.104277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50432)
0808 23:45:30.104434 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.104519 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:45:30.104895 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
I20260808 23:45:30.207700 11167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 4
I20260808 23:45:30.208992 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.208387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50446)
0808 23:45:30.208652 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.208777 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 23:45:30.210104 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.209713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50462)
0808 23:45:30.209870 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.209955 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 23:45:30.210323 11167 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
I20260808 23:45:30.211112 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 3
I20260808 23:45:30.212340 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.211753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50470)
0808 23:45:30.212001 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.212120 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 23:45:30.213408 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.213004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50478)
0808 23:45:30.213192 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.213272 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260808 23:45:30.213629 11084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:33435: Network error: Client connection negotiation failed: client connection to 127.18.186.188:33435: connect: Connection refused (error 111)
W20260808 23:45:30.219751 11175 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:45:30.220047 11175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:30.220119 11175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:30.220201 11175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:30.227627 11175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:30.227761 11175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:30.227872 11175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:30.227957 11175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:30.228065 11175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:30.238219 11175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:43921,127.18.186.189:46167,127.18.186.188:33435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab
--principal=kudu/127.18.186.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.188:33435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.186.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:30.239638 11175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:30.240923 11175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:30.253031 11183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:30.253181 11180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:30.253119 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:30.254251 11175 server_base.cc:1034] running on GCE node
I20260808 23:45:30.255115 11175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:30.256639 11175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:30.257892 11175 hybrid_clock.cc:648] HybridClock initialized: now 1786232730257809 us; error 34 us; skew 500 ppm
I20260808 23:45:30.258229 11175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232730, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:30.264875 11175 init.cc:377] Logged in from keytab as kudu/127.18.186.188@KRBTEST.COM (short username kudu)
I20260808 23:45:30.266309 11175 webserver.cc:460] Webserver started at http://127.18.186.188:39797/ using document root <none> and password file <none>
I20260808 23:45:30.267021 11175 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:30.267138 11175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:30.267562 11175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:30.270566 11175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/data/instance:
uuid: "8010356bff95431f8b56b8a1d38e685e"
format_stamp: "Formatted at 2026-08-08 23:45:30 on dist-test-slave-5k54"
server_key: "4c0417dc1b82f657b7f2b33ea98c963b"
server_key_iv: "bd31ab46d5fad4753a24ce777c2c9bf2"
server_key_version: "encryptionkey@0"
I20260808 23:45:30.271359 11175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/wal/instance:
uuid: "8010356bff95431f8b56b8a1d38e685e"
format_stamp: "Formatted at 2026-08-08 23:45:30 on dist-test-slave-5k54"
server_key: "4c0417dc1b82f657b7f2b33ea98c963b"
server_key_iv: "bd31ab46d5fad4753a24ce777c2c9bf2"
server_key_version: "encryptionkey@0"
I20260808 23:45:30.276664 11175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 23:45:30.280354 11190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:30.281662 11175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:45:30.281840 11175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/wal
uuid: "8010356bff95431f8b56b8a1d38e685e"
format_stamp: "Formatted at 2026-08-08 23:45:30 on dist-test-slave-5k54"
server_key: "4c0417dc1b82f657b7f2b33ea98c963b"
server_key_iv: "bd31ab46d5fad4753a24ce777c2c9bf2"
server_key_version: "encryptionkey@0"
I20260808 23:45:30.282084 11175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:30.300448 11175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:30.304301 11175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:30.304575 11175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:30.323905 11175 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.188:33435
I20260808 23:45:30.323876 11245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.188:33435 every 8 connection(s)
I20260808 23:45:30.325307 11175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
I20260808 23:45:30.329391 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:30.330080 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 11175
I20260808 23:45:30.330292 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786232366675349-19178-0/minicluster-data/master-2/wal/instance
I20260808 23:45:30.330735 19178 external_mini_cluster.cc:1428] Setting key 662e3df631a8dc7d9dd8991483a6bc11
I20260808 23:45:30.338497 11246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232730, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:30.354476 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.332745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50492 (local address 127.18.186.188:33435)
0808 23:45:30.333543 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.333554 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.333601 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:45:30.335412 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.335485 (+ 73us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.335897 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.336221 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.338025 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.339396 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.341153 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.342323 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.342642 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.346840 (+ 4198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.346848 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.346860 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.346895 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.350295 (+ 3400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.350953 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.350968 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.350975 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.351120 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.351549 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.351553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.351556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.352090 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.352429 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.352854 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.353204 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":377,"threads_started":1}
I20260808 23:45:30.356655 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.341362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:43921 (local address 127.0.0.1:48440)
0808 23:45:30.342170 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.342228 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:30.342417 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:30.342912 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:30.342918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:30.343700 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:30.344061 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.344070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.345082 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.345094 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.346170 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.346182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.346936 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.346948 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.347123 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.347946 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:30.347964 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:30.350448 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:30.353870 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.353879 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.353885 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.354448 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.354945 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.354948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.354951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.355072 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.355765 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:30.355771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:30.355943 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 23:45:30.356107 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 23:45:30.356361 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":420,"threads_started":1}
I20260808 23:45:30.356885 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.341288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.18.186.190:43921)
0808 23:45:30.341795 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.341803 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.342435 (+ 632us) server_negotiation.cc:407] Connection header received
0808 23:45:30.342553 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.342557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.342722 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.342844 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.344209 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.344928 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.346320 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.346734 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.346867 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.350638 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.350643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.350646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.350683 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.353736 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.354608 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.354616 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.354621 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.354820 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.355190 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.355197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.355202 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.355625 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.355786 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.356249 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.356502 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":262,"threads_started":1}
I20260808 23:45:30.358304 11246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232730, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:30.371665 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.358973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:46167 (local address 127.0.0.1:42792)
0808 23:45:30.359124 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.359144 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:30.359236 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:30.359699 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:30.359705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:30.360193 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:30.360395 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.360410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.361369 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.361374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.362299 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.362310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.363077 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.363083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.363278 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.363848 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:30.363868 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:30.365895 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:30.369264 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.369270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.369274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.369636 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.370006 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.370011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.370015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.370156 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.370668 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:30.370673 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:30.370830 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 23:45:30.370977 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 23:45:30.371249 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 23:45:30.371696 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.359215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.18.186.189:46167)
0808 23:45:30.359381 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.359385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.359400 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:45:30.359500 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.359503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.359560 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.359640 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.360543 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.361226 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.362458 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.362924 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.363018 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.366078 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.366082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.366085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.366115 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.369130 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.369753 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.369760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.369763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.369899 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.370263 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.370268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.370271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.370550 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.370709 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.371079 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.371279 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:45:30.372617 11246 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:45:30.372983 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.358358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.18.186.190:43921)
0808 23:45:30.358507 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.358513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.358554 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:45:30.358749 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.358754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.358891 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.360592 (+ 1701us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.360707 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.361393 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.362566 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.362991 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.363113 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.367338 (+ 4225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.367342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.367346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.367376 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.370063 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.370729 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.370737 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.370740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.370875 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.371255 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.371261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.371264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.371620 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.372324 (+ 704us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.372468 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.372703 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232730, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:45:30.386121 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.373730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.18.186.188:33435)
0808 23:45:30.373881 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.373885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.373897 (+ 12us) server_negotiation.cc:407] Connection header received
0808 23:45:30.373929 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.373932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.373977 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.374091 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.374910 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.375646 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.376748 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.377146 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.377285 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.380543 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.380548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.380551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.380579 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.383758 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.384360 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.384371 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.384373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.384473 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.384828 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.384834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.384838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.385102 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.385249 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.385569 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.385798 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:45:30.386821 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.373385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50502)
0808 23:45:30.373548 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.373568 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:30.373665 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:30.374126 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:30.374129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:30.374582 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:30.374761 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.374781 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.375768 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.375774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.376616 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.376626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.377300 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.377305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.377522 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.378142 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:30.378158 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:30.380375 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:30.383883 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.383886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.383889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.384229 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.384575 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.384580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.384584 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.384708 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.385213 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:30.385217 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:30.385350 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 23:45:30.385496 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 23:45:30.386005 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":153}
I20260808 23:45:30.389544 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.374673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.18.186.189:46167)
0808 23:45:30.374826 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.374831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.374847 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:45:30.375021 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.375027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.375087 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.375185 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.376421 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.377090 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.378476 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.378902 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.379036 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.383504 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.383510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.383514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.383555 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.386904 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.387698 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.387703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.387706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.387884 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.388316 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.388325 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.388327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.388578 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.388801 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.389126 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.389286 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:45:30.390828 11084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 4
I20260808 23:45:30.392937 11246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e: Bootstrap starting.
I20260808 23:45:30.394611 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:30.397154 11246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:30.398524 11246 log.cc:824] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
I20260808 23:45:30.401816 11246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e: No bootstrap required, opened a new log
I20260808 23:45:30.398365 11059 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9a45097c1a6843889e78106749f7a96a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446
I20260808 23:45:30.407289 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.391496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50506)
0808 23:45:30.391943 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.391967 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:30.392112 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:30.392539 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:30.392544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:30.393072 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:30.393295 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.393322 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.394627 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.394635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.395786 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.395798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.396582 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.396588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.396792 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.397397 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:30.397415 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:30.400172 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:30.404452 (+ 4280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.404457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.404463 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.405015 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.405445 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.405452 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.405457 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.405615 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.406294 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:30.406299 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:30.406460 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 23:45:30.406600 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 23:45:30.406893 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305}
I20260808 23:45:30.407374 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.391528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50506 (local address 127.18.186.188:33435)
0808 23:45:30.391699 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.391704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.392047 (+ 343us) server_negotiation.cc:407] Connection header received
0808 23:45:30.392232 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.392237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.392332 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.392516 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.393464 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.394421 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.395939 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.396418 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.396544 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.400369 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.400376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.400382 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.400419 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.404310 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.405155 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.405161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.405165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.405325 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.405733 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.405737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.405741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.406160 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.406320 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.406748 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.406977 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:45:30.412599 11246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } }
I20260808 23:45:30.413065 11084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a: Bootstrap starting.
I20260808 23:45:30.413056 11246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8010356bff95431f8b56b8a1d38e685e, State: Initialized, Role: FOLLOWER
I20260808 23:45:30.413554 11246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } }
I20260808 23:45:30.414521 11253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.414917 11253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:30.415805 11246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:30.417136 11084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:30.418628 11084 log.cc:824] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:30.422458 11084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a: No bootstrap required, opened a new log
I20260808 23:45:30.430263 11084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } }
W20260808 23:45:30.430614 11265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 23:45:30.430706 11084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a45097c1a6843889e78106749f7a96a, State: Initialized, Role: FOLLOWER
W20260808 23:45:30.430786 11265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:45:30.431191 11084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } }
I20260808 23:45:30.438155 11267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.438536 11267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:30.428002 11175 master_runner.cc:186] Error getting master registration for 127.18.186.190:43921: OK, instance_id { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" instance_seqno: 1786232729252859 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:45:30.439299 11175 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:45:30.440465 11084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:30.440589 11059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 23:45:30.440812 11059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:30.443722 11059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } }
I20260808 23:45:30.445233 11059 leader_election.cc:288] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [CANDIDATE]: Term 1 election: Requested vote from peers 5c667a7edc9f42a7a607efc8cfd7ac02 (127.18.186.189:46167), 8010356bff95431f8b56b8a1d38e685e (127.18.186.188:33435)
I20260808 23:45:30.446301 11219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a45097c1a6843889e78106749f7a96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8010356bff95431f8b56b8a1d38e685e"
I20260808 23:45:30.446689 11219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:30.449443 11219 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a45097c1a6843889e78106749f7a96a in term 1.
I20260808 23:45:30.450101 11032 leader_election.cc:302] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8010356bff95431f8b56b8a1d38e685e, 9a45097c1a6843889e78106749f7a96a; no voters:
I20260808 23:45:30.450672 11267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:45:30.451387 11267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [term 1 LEADER]: Becoming Leader. State: Replica: 9a45097c1a6843889e78106749f7a96a, State: Running, Role: LEADER
I20260808 23:45:30.452015 11267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } }
I20260808 23:45:30.455538 11272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a45097c1a6843889e78106749f7a96a. Latest consensus state: current_term: 1 leader_uuid: "9a45097c1a6843889e78106749f7a96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.456060 11272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:30.457237 11276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:30.463603 11276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 23:45:30.492370 11037 tablet.cc:2369] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:45:30.497313 11219 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [term 1 FOLLOWER]: Refusing update from remote peer 9a45097c1a6843889e78106749f7a96a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:45:30.498845 11272 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:45:30.509534 11167 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } attempt: 5
I20260808 23:45:30.529959 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.483403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42800 (local address 127.18.186.189:46167)
0808 23:45:30.483961 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.483969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.489436 (+ 5467us) server_negotiation.cc:407] Connection header received
0808 23:45:30.492102 (+ 2666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.492109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.492207 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.492310 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.499571 (+ 7261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.500756 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.502874 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.503640 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.503789 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.518396 (+ 14607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.518405 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.518411 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.518471 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.522945 (+ 4474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.526791 (+ 3846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.526798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.526804 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.527001 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.527495 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.527500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.527503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.527928 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.528191 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.529282 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.529544 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379}
I20260808 23:45:30.540431 11253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a45097c1a6843889e78106749f7a96a. Latest consensus state: current_term: 1 leader_uuid: "9a45097c1a6843889e78106749f7a96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.540726 11253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:30.523751 11015 master_runner.cc:186] Error getting master registration for 127.18.186.189:46167: OK, instance_id { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" instance_seqno: 1786232729779857 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:45:30.543370 11015 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:45:30.543370 11272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a45097c1a6843889e78106749f7a96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.543797 11272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:30.544294 11281 mvcc.cc:205] Tried to move back new op lower bound from 7316409264094404608 to 7316409263936405504. Current Snapshot: MvccSnapshot[applied={T|T < 7316409264094404608}]
I20260808 23:45:30.545001 11276 catalog_manager.cc:1347] Generated new cluster ID: 787478432e74412a9c9642b7e78fcf82
I20260808 23:45:30.545094 11276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 23:45:30 dist-test-slave-5k54 krb5kdc[11002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232729, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
I20260808 23:45:30.548720 11290 mvcc.cc:205] Tried to move back new op lower bound from 7316409264094404608 to 7316409263936405504. Current Snapshot: MvccSnapshot[applied={T|T < 7316409264094404608}]
I20260808 23:45:30.557979 11253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a45097c1a6843889e78106749f7a96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.558282 11253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:30.559691 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.510195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:33435 (local address 127.0.0.1:50528)
0808 23:45:30.510608 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:30.510630 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:30.510749 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:30.536300 (+ 25551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:30.536307 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:30.536858 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:30.537093 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.537108 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.540600 (+ 3492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.540607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.541884 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:30.541900 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.543917 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.543924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:30.544193 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.545033 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:30.545060 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:30.548110 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:30.554399 (+ 6289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.554424 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.554429 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.555074 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.555738 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:30.555743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:30.555747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:30.556042 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:30.558677 (+ 2635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:30.558682 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:30.558838 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 23:45:30.558982 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 23:45:30.559265 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260808 23:45:30.562806 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.510966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50528 (local address 127.18.186.188:33435)
0808 23:45:30.535309 (+ 24343us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.535317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.535340 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:45:30.535600 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.535605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.535879 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.537406 (+ 1527us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.539060 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.540371 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.542167 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.543652 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.544649 (+ 997us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.548440 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.548448 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.548452 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.548511 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.554197 (+ 5686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.555277 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.555282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.555285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.555550 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.556225 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.556230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.556234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.558537 (+ 2303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.558681 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.559757 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.560409 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23811,"thread_start_us":23065,"threads_started":1}
I20260808 23:45:30.569732 11167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02: Bootstrap starting.
I20260808 23:45:30.578179 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.483071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.18.186.188:33435)
0808 23:45:30.483272 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.483279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.523340 (+ 40061us) server_negotiation.cc:407] Connection header received
0808 23:45:30.527983 (+ 4643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.527992 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.528104 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.528193 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.533667 (+ 5474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.534905 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.537105 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.538204 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.538334 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.565922 (+ 27588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.565929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.565934 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.565972 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.569669 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.570554 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.570560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.570564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.570750 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.571228 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.571233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.571236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.571604 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.571847 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.577510 (+ 5663us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.577776 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:45:30.581568 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.483075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48454 (local address 127.18.186.190:43921)
0808 23:45:30.483267 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.483275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.523564 (+ 40289us) server_negotiation.cc:407] Connection header received
0808 23:45:30.530433 (+ 6869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.530441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.530539 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.530618 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.536933 (+ 6315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.538077 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.560702 (+ 22625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.561461 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.561569 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.572441 (+ 10872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:30.572451 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:30.572457 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:30.572525 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:30.576505 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.578922 (+ 2417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.578928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.578931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.579096 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:30.579606 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:30.579614 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:30.579620 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:30.580009 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:30.580274 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.580825 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.581118 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:45:30.583621 11167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:30.585319 11167 log.cc:824] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:30.596035 11167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02: No bootstrap required, opened a new log
I20260808 23:45:30.607385 11167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } }
I20260808 23:45:30.607882 11167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c667a7edc9f42a7a607efc8cfd7ac02, State: Initialized, Role: FOLLOWER
I20260808 23:45:30.608440 11167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } }
I20260808 23:45:30.614418 11301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.615271 11301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:30.615743 11167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:30.615839 11138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a45097c1a6843889e78106749f7a96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02"
I20260808 23:45:30.616304 11138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:30.626678 11138 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a45097c1a6843889e78106749f7a96a in term 1.
I20260808 23:45:30.627672 11137 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [term 1 FOLLOWER]: Refusing update from remote peer 9a45097c1a6843889e78106749f7a96a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:45:30.630127 11272 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:45:30.661325 11301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a45097c1a6843889e78106749f7a96a. Latest consensus state: current_term: 1 leader_uuid: "9a45097c1a6843889e78106749f7a96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.661798 11301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:30.666880 11276 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:30.668678 11267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a45097c1a6843889e78106749f7a96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.669019 11276 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:30.669123 11267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:30.672619 11311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a45097c1a6843889e78106749f7a96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a45097c1a6843889e78106749f7a96a" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 43921 } } peers { permanent_uuid: "5c667a7edc9f42a7a607efc8cfd7ac02" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 46167 } } peers { permanent_uuid: "8010356bff95431f8b56b8a1d38e685e" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 33435 } } }
I20260808 23:45:30.673295 11311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:30.687589 11276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a45097c1a6843889e78106749f7a96a: Generated new TSK 0
I20260808 23:45:30.700544 11276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:30.722287 11323 catalog_manager.cc:1259] Loaded cluster ID: 787478432e74412a9c9642b7e78fcf82
I20260808 23:45:30.722402 11323 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02: loading cluster ID for follower catalog manager: success
I20260808 23:45:30.726563 11323 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02: acquiring CA information for follower catalog manager: success
I20260808 23:45:30.739655 11323 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5c667a7edc9f42a7a607efc8cfd7ac02: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 23:45:30.854179 11250 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:30.845484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.18.186.190:43921)
0808 23:45:30.845739 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.845747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.852109 (+ 6362us) server_negotiation.cc:407] Connection header received
0808 23:45:30.852424 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.852431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.852519 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.852604 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.853932 (+ 1328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48464: BlockingRecv error: recv got EOF from 127.0.0.1:48464 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 23:45:30.856808 11172 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:30.845042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42804 (local address 127.18.186.189:46167)
0808 23:45:30.845295 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.845304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.854927 (+ 9623us) server_negotiation.cc:407] Connection header received
0808 23:45:30.855208 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.855215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.855315 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.855399 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.856529 (+ 1130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42804: BlockingRecv error: recv got EOF from 127.0.0.1:42804 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 23:45:30.864717 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:30.844566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.18.186.188:33435)
0808 23:45:30.844799 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.844805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.857617 (+ 12812us) server_negotiation.cc:407] Connection header received
0808 23:45:30.857913 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.857919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.858011 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.858091 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.864409 (+ 6318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50538: BlockingRecv error: recv got EOF from 127.0.0.1:50538 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:45:30.918666 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:30.910397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42816 (local address 127.18.186.189:46167)
0808 23:45:30.910594 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.910600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.910625 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:45:30.910939 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.910945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.911168 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.911339 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:30.912113 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.913301 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.915414 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:30.916134 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:30.916232 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:30.917103 (+ 871us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:30.917815 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:30.917934 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:30.918184 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 23:45:30.918302 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 23:45:30.982148 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:30.970659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.18.186.188:33435)
0808 23:45:30.970919 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:30.970929 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:30.980109 (+ 9180us) server_negotiation.cc:407] Connection header received
0808 23:45:30.980483 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:30.980490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:30.980636 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:30.980721 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:30.981893 (+ 1172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50540: BlockingRecv error: recv got EOF from 127.0.0.1:50540 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:45:31.037345 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.018044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.18.186.189:46167)
0808 23:45:31.018289 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.018296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.030146 (+ 11850us) server_negotiation.cc:407] Connection header received
0808 23:45:31.030691 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.030698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.030799 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.030884 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.031639 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.032830 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.034917 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.035663 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.035769 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.036440 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.036590 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.036659 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.036911 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.036998 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 23:45:31.088656 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:31.076358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50542 (local address 127.18.186.188:33435)
0808 23:45:31.076540 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.076546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.086712 (+ 10166us) server_negotiation.cc:407] Connection header received
0808 23:45:31.087031 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.087042 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.087153 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.087234 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:31.088382 (+ 1148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50542: BlockingRecv error: recv got EOF from 127.0.0.1:50542 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:45:31.141397 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.123872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.18.186.189:46167)
0808 23:45:31.124089 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.124096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.134001 (+ 9905us) server_negotiation.cc:407] Connection header received
0808 23:45:31.134539 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.134545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.134645 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.134722 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.135461 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.136613 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.138717 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.139566 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.139743 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.140252 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.140506 (+ 254us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.140611 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.140839 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.140987 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 23:45:31.155406 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:31.153158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.18.186.188:33435)
0808 23:45:31.153342 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.153349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.153822 (+ 473us) server_negotiation.cc:407] Connection header received
0808 23:45:31.154078 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.154085 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.154174 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.154242 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:31.155182 (+ 940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50550: BlockingRecv error: recv got EOF from 127.0.0.1:50550 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:45:31.178751 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.169524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42850 (local address 127.18.186.189:46167)
0808 23:45:31.169714 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.169721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.170335 (+ 614us) server_negotiation.cc:407] Connection header received
0808 23:45:31.170547 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.170555 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.170665 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.170793 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.171311 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.172589 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.176250 (+ 3661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.177102 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.177246 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.177757 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.177934 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.178044 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.178228 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.178352 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 23:45:31.189230 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:31.186612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.18.186.188:33435)
0808 23:45:31.186856 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.186864 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.187515 (+ 651us) server_negotiation.cc:407] Connection header received
0808 23:45:31.187857 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.187863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.187966 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.188081 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:31.188985 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50560: BlockingRecv error: recv got EOF from 127.0.0.1:50560 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 23:45:31.208971 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.202025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42852 (local address 127.18.186.189:46167)
0808 23:45:31.202217 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.202223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.202698 (+ 475us) server_negotiation.cc:407] Connection header received
0808 23:45:31.202913 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.202920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.203014 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.203127 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.203629 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.204919 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.206819 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.207541 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.207681 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.208136 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.208245 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.208321 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.208500 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.208565 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 23:45:31.218250 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:31.215947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50576 (local address 127.18.186.188:33435)
0808 23:45:31.216207 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.216213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.216572 (+ 359us) server_negotiation.cc:407] Connection header received
0808 23:45:31.216740 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.216744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.216820 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.216914 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:31.218023 (+ 1109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50576: BlockingRecv error: recv got EOF from 127.0.0.1:50576 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
I20260808 23:45:31.244556 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.237758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.18.186.189:46167)
0808 23:45:31.237978 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.237985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.238414 (+ 429us) server_negotiation.cc:407] Connection header received
0808 23:45:31.238578 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.238586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.238728 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.238850 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.239281 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.240597 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.242168 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.242841 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.242994 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.243570 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.243742 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.243839 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.244102 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.244212 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 23:45:31.261992 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:31.259523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.18.186.188:33435)
0808 23:45:31.259729 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.259737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.260365 (+ 628us) server_negotiation.cc:407] Connection header received
0808 23:45:31.260574 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.260580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.260689 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.260814 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:31.261771 (+ 957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50590: BlockingRecv error: recv got EOF from 127.0.0.1:50590 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 23:45:31.304342 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.297791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42864 (local address 127.18.186.189:46167)
0808 23:45:31.298064 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.298071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.298475 (+ 404us) server_negotiation.cc:407] Connection header received
0808 23:45:31.298677 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.298684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.298788 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.298919 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.299449 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.300611 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.302109 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.302861 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.303002 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.303446 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.303611 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.303721 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.303863 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.303959 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260808 23:45:31.313084 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:31.311170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.18.186.188:33435)
0808 23:45:31.311360 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.311366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.311765 (+ 399us) server_negotiation.cc:407] Connection header received
0808 23:45:31.311925 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.311931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.312024 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.312130 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:31.312853 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50594: BlockingRecv error: recv got EOF from 127.0.0.1:50594 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:45:31.385645 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.380480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.18.186.189:46167)
0808 23:45:31.380656 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.380660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.381069 (+ 409us) server_negotiation.cc:407] Connection header received
0808 23:45:31.381218 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.381224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.381294 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.381408 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.381922 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.382638 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.384048 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.384476 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.384573 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.384983 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.385077 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.385145 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.385286 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.385352 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 23:45:31.393862 11249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:31.391925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50600 (local address 127.18.186.188:33435)
0808 23:45:31.392165 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.392171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.392447 (+ 276us) server_negotiation.cc:407] Connection header received
0808 23:45:31.392669 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.392673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.392741 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.392846 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:31.393652 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50600: BlockingRecv error: recv got EOF from 127.0.0.1:50600 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:45:31.432839 11265 catalog_manager.cc:1259] Loaded cluster ID: 787478432e74412a9c9642b7e78fcf82
I20260808 23:45:31.433024 11265 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e: loading cluster ID for follower catalog manager: success
I20260808 23:45:31.435555 11265 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e: acquiring CA information for follower catalog manager: success
I20260808 23:45:31.437310 11265 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8010356bff95431f8b56b8a1d38e685e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:45:31.530552 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.525564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.18.186.189:46167)
0808 23:45:31.525848 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.525854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.526131 (+ 277us) server_negotiation.cc:407] Connection header received
0808 23:45:31.526313 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.526317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.526404 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.526486 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.526874 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.527598 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.528928 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.529335 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.529436 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.529808 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.529914 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.529982 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.530118 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.530179 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 23:45:31.541546 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.536067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50608 (local address 127.18.186.188:33435)
0808 23:45:31.536352 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.536358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.536555 (+ 197us) server_negotiation.cc:407] Connection header received
0808 23:45:31.536678 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.536682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.536872 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.536974 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.537316 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.538020 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.539300 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.539790 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.539882 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.540407 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.540947 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.541062 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.541149 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.541211 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 23:45:31.566781 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.547693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.18.186.190:43921)
0808 23:45:31.559239 (+ 11546us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.559248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.559278 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:45:31.559479 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.559487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.559764 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.559990 (+ 226us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.561274 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.562399 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.564167 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.564599 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.564739 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.565266 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.565595 (+ 329us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.566447 (+ 852us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.566486 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.566555 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11004,"spinlock_wait_cycles":275840,"thread_start_us":11244,"threads_started":1}
I20260808 23:45:31.568035 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.548604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42900 (local address 127.18.186.189:46167)
0808 23:45:31.548831 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.548850 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.561156 (+ 12306us) server_negotiation.cc:407] Connection header received
0808 23:45:31.561363 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.561371 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.561467 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.561555 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.562624 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.563341 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.565813 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.566401 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.566501 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.567195 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.567332 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.567395 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.567667 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.567750 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:45:31.568164 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:31.548778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.18.186.188:33435)
0808 23:45:31.549341 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:31.549349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:31.561444 (+ 12095us) server_negotiation.cc:407] Connection header received
0808 23:45:31.561581 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:31.561586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:31.561671 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:31.561758 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:31.562283 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.563315 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.565769 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:31.566292 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:31.566394 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:31.566905 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:31.567030 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:31.567118 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:31.567821 (+ 703us) server_negotiation.cc:299] Negotiation successful
0808 23:45:31.567909 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395}
I20260808 23:45:31.573057 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 11015
I20260808 23:45:31.574321 11079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:31.685848 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 11015
I20260808 23:45:31.711138 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 11092
I20260808 23:45:31.712419 11162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:31.821132 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 11092
I20260808 23:45:31.844488 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 11175
I20260808 23:45:31.845712 11240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:31.952647 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 11175
2026-08-08T23:45:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4657 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:45:32 dist-test-slave-5k54 krb5kdc[11442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:45:32 dist-test-slave-5k54 krb5kdc[11442](info): set up 2 sockets
Aug 08 23:45:32 dist-test-slave-5k54 krb5kdc[11442](info): commencing operation
krb5kdc: starting...
W20260808 23:45:33.143186 19178 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/test-user.keytab.
Aug 08 23:45:33 dist-test-slave-5k54 krb5kdc[11442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:45:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:45:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.186.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
Entry for principal kudu/127.18.186.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab.
I20260808 23:45:33.467166 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42181
--webserver_interface=127.18.186.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42181,127.18.186.189:32959,127.18.186.188:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:45:33.810035 11456 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:45:33.810393 11456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:33.810518 11456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:33.810647 11456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:33.817991 11456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:33.818082 11456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:33.818179 11456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:33.818264 11456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:33.818372 11456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:33.828482 11456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42181,127.18.186.189:32959,127.18.186.188:38925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.190.keytab
--principal=kudu/127.18.186.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.190:42181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.186.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:33.829816 11456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:33.831182 11456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:33.842818 11461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:33.842935 11462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:33.842969 11464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:33.844081 11456 server_base.cc:1034] running on GCE node
I20260808 23:45:33.844954 11456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:33.846462 11456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:33.847764 11456 hybrid_clock.cc:648] HybridClock initialized: now 1786232733847648 us; error 85 us; skew 500 ppm
I20260808 23:45:33.848127 11456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:33 dist-test-slave-5k54 krb5kdc[11442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:33.851523 11456 init.cc:377] Logged in from keytab as kudu/127.18.186.190@KRBTEST.COM (short username kudu)
I20260808 23:45:33.852996 11456 webserver.cc:460] Webserver started at http://127.18.186.190:44145/ using document root <none> and password file <none>
I20260808 23:45:33.853713 11456 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:33.853847 11456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:33.854174 11456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:33.857240 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/data/instance:
uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7"
format_stamp: "Formatted at 2026-08-08 23:45:33 on dist-test-slave-5k54"
server_key: "6e0259bdb96a7842113913fac150eebf"
server_key_iv: "eeda62c9b4e5b79430834eaeeb16605f"
server_key_version: "encryptionkey@0"
I20260808 23:45:33.858139 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/wal/instance:
uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7"
format_stamp: "Formatted at 2026-08-08 23:45:33 on dist-test-slave-5k54"
server_key: "6e0259bdb96a7842113913fac150eebf"
server_key_iv: "eeda62c9b4e5b79430834eaeeb16605f"
server_key_version: "encryptionkey@0"
I20260808 23:45:33.863394 11456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 23:45:33.866777 11471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:33.868142 11456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:45:33.868325 11456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/wal
uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7"
format_stamp: "Formatted at 2026-08-08 23:45:33 on dist-test-slave-5k54"
server_key: "6e0259bdb96a7842113913fac150eebf"
server_key_iv: "eeda62c9b4e5b79430834eaeeb16605f"
server_key_version: "encryptionkey@0"
I20260808 23:45:33.868549 11456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:33.891937 11456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:33.896018 11456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:33.896310 11456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:33.915189 11456 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.190:42181
I20260808 23:45:33.915231 11524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.190:42181 every 8 connection(s)
I20260808 23:45:33.916625 11456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/data/info.pb
I20260808 23:45:33.919065 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 11456
I20260808 23:45:33.919258 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-0/wal/instance
I20260808 23:45:33.919751 19178 external_mini_cluster.cc:1428] Setting key 44287397934052683b1339d0eb7ac495
I20260808 23:45:33.920679 11525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:33.927206 11525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:45:33 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:33.953182 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:33.930598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42181 (local address 127.0.0.1:42460)
0808 23:45:33.931398 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:33.931484 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:33.933769 (+ 2285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:33.934708 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:33.934713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:33.935295 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:33.935736 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:33.935754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:33.938077 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:33.938080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:33.939216 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:33.939228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:33.940618 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:33.940624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:33.940759 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:33.941502 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:33.941523 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:33.944004 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:33.948688 (+ 4684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:33.948699 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:33.948709 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:33.949181 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:33.949709 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:33.949713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:33.949735 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:33.950019 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:33.951158 (+ 1139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:33.951164 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:33.951401 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 23:45:33.951688 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 23:45:33.952114 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":299,"threads_started":1}
I20260808 23:45:33.953235 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:33.931688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.18.186.190:42181)
0808 23:45:33.932348 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:33.932354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:33.932399 (+ 45us) server_negotiation.cc:407] Connection header received
0808 23:45:33.934145 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:33.934156 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:33.934473 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:33.934679 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:33.935941 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:33.937923 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:33.939368 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:33.940407 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:33.940600 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:33.944296 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:33.944310 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:33.944329 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:33.944397 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:33.948542 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:33.949336 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:33.949347 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:33.949353 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:33.949561 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:33.950172 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:33.950180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:33.950183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:33.950994 (+ 811us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:33.951240 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:33.951915 (+ 675us) server_negotiation.cc:299] Negotiation successful
0808 23:45:33.952321 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":475,"thread_start_us":340,"threads_started":1}
WARNING: no policy specified for kudu/127.18.186.189@KRBTEST.COM; defaulting to no policy
I20260808 23:45:33.956333 11525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:45:33.957381 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:33.956990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:36932)
0808 23:45:33.957158 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:33.957269 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 23:45:33.957621 11474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.189@KRBTEST.COM" created.
I20260808 23:45:33.959209 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:33.958795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:36942)
0808 23:45:33.958960 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:33.959032 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 23:45:33.959455 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:32959: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab.
Entry for principal kudu/127.18.186.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab.
I20260808 23:45:33.996771 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.189:32959
--webserver_interface=127.18.186.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab
--principal=kudu/127.18.186.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42181,127.18.186.189:32959,127.18.186.188:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:45:34.008834 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } attempt: 1
I20260808 23:45:34.010143 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.009656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:36946)
0808 23:45:34.009886 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.010001 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 23:45:34.011152 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.010799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:36954)
0808 23:45:34.010956 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.011017 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 23:45:34.011402 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:32959: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
I20260808 23:45:34.079699 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } attempt: 2
I20260808 23:45:34.080914 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.080393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:36956)
0808 23:45:34.080652 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.080741 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 23:45:34.082031 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.081677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:36970)
0808 23:45:34.081826 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.081912 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 23:45:34.082222 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:32959: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
I20260808 23:45:34.173442 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } attempt: 3
I20260808 23:45:34.174567 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.174124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:36972)
0808 23:45:34.174336 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.174434 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 23:45:34.175688 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.175266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:36980)
0808 23:45:34.175493 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.175556 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 23:45:34.175894 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:32959: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
I20260808 23:45:34.319141 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } attempt: 4
I20260808 23:45:34.320451 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.319796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:57850)
0808 23:45:34.320148 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.320260 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260808 23:45:34.321564 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.321132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:57866)
0808 23:45:34.321287 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.321398 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 23:45:34.321766 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.189:32959: Network error: Client connection negotiation failed: client connection to 127.18.186.189:32959: connect: Connection refused (error 111)
W20260808 23:45:34.338783 11534 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:45:34.339112 11534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:34.339187 11534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:34.339268 11534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:45:34.346480 11534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:34.346562 11534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:34.346618 11534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:34.346670 11534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:34.346731 11534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:34.356622 11534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42181,127.18.186.189:32959,127.18.186.188:38925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.189.keytab
--principal=kudu/127.18.186.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.189:32959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.186.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:34.357774 11534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:34.359127 11534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:34.370426 11539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:34.370456 11540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:34.370618 11542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:34.371747 11534 server_base.cc:1034] running on GCE node
I20260808 23:45:34.372501 11534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:34.373998 11534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:34.375221 11534 hybrid_clock.cc:648] HybridClock initialized: now 1786232734375146 us; error 54 us; skew 500 ppm
I20260808 23:45:34.375624 11534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:34.379040 11534 init.cc:377] Logged in from keytab as kudu/127.18.186.189@KRBTEST.COM (short username kudu)
I20260808 23:45:34.380450 11534 webserver.cc:460] Webserver started at http://127.18.186.189:36947/ using document root <none> and password file <none>
I20260808 23:45:34.381152 11534 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:34.381285 11534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:34.381699 11534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:34.384701 11534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/data/instance:
uuid: "b72a6042aa994fba88a9ce710ebbcb4b"
format_stamp: "Formatted at 2026-08-08 23:45:34 on dist-test-slave-5k54"
server_key: "bdfb656ba6c7079e73aa73825c281c09"
server_key_iv: "e22021807a9cddd5506b76c0aaea3fe5"
server_key_version: "encryptionkey@0"
I20260808 23:45:34.385525 11534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/wal/instance:
uuid: "b72a6042aa994fba88a9ce710ebbcb4b"
format_stamp: "Formatted at 2026-08-08 23:45:34 on dist-test-slave-5k54"
server_key: "bdfb656ba6c7079e73aa73825c281c09"
server_key_iv: "e22021807a9cddd5506b76c0aaea3fe5"
server_key_version: "encryptionkey@0"
I20260808 23:45:34.390625 11534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:45:34.394032 11549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:34.395324 11534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:45:34.395535 11534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/wal
uuid: "b72a6042aa994fba88a9ce710ebbcb4b"
format_stamp: "Formatted at 2026-08-08 23:45:34 on dist-test-slave-5k54"
server_key: "bdfb656ba6c7079e73aa73825c281c09"
server_key_iv: "e22021807a9cddd5506b76c0aaea3fe5"
server_key_version: "encryptionkey@0"
I20260808 23:45:34.395746 11534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:34.408208 11534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:34.412127 11534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:34.412391 11534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:34.431603 11534 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.189:32959
I20260808 23:45:34.431640 11604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.189:32959 every 8 connection(s)
I20260808 23:45:34.433014 11534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/data/info.pb
I20260808 23:45:34.436767 11605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:34.439672 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 11534
I20260808 23:45:34.439873 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-1/wal/instance
I20260808 23:45:34.440457 19178 external_mini_cluster.cc:1428] Setting key 97d14f418ced2db4598059a876023623
I20260808 23:45:34.445636 11605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:34.468421 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.449024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35262 (local address 127.18.186.190:42181)
0808 23:45:34.449268 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:34.449277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:34.450383 (+ 1106us) server_negotiation.cc:407] Connection header received
0808 23:45:34.452492 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:34.452500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:34.452597 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:34.452721 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:34.454212 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.455304 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.457749 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.458178 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.458273 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.462450 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:34.462455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:34.462458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:34.462493 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:34.465328 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.466078 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.466091 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.466093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.466210 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.466646 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.466650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.466655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.466932 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:34.467084 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:34.467841 (+ 757us) server_negotiation.cc:299] Negotiation successful
0808 23:45:34.468097 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 23:45:34.469394 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.449411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42181 (local address 127.0.0.1:35262)
0808 23:45:34.450205 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.450257 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:34.452223 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:34.452907 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:34.452916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:34.453440 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:34.454038 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.454055 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.455492 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.455499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.457606 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.457623 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.458399 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.458416 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.458739 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.459524 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:34.459563 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:34.462212 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:34.465479 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:34.465488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:34.465503 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:34.465946 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:34.466322 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:34.466327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:34.466331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:34.466528 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:34.467058 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:34.467065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:34.467459 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 23:45:34.467736 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 23:45:34.468254 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":333,"threads_started":1}
I20260808 23:45:34.471633 11605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.186.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.186.188@KRBTEST.COM" created.
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:45:34.488662 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.472382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:57878)
0808 23:45:34.472591 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.472610 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:34.472717 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:34.474060 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:34.474063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:34.474443 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:34.474569 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.474577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.476486 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.476492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.477571 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.477584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.478685 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.478691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.478901 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.479510 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:34.479537 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:34.482024 (+ 2487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:34.486247 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:34.486251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:34.486253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:34.486599 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:34.487002 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:34.487005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:34.487007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:34.487136 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:34.487899 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:34.487903 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:34.488016 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 23:45:34.488123 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 23:45:34.488341 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 23:45:34.489137 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.472679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57878 (local address 127.18.186.189:32959)
0808 23:45:34.473517 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:34.473526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:34.473561 (+ 35us) server_negotiation.cc:407] Connection header received
0808 23:45:34.473628 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:34.473636 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:34.473887 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:34.474043 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:34.474764 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.476279 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.477699 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.478531 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.478626 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.482253 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:34.482270 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:34.482289 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:34.482331 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:34.486107 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.486728 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.486751 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.486756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.486898 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.487231 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.487238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.487244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.487772 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:34.487978 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:34.488313 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 23:45:34.488589 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":498,"threads_started":1}
I20260808 23:45:34.491462 11605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:45:34.492843 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.492202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53550)
0808 23:45:34.492437 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.492612 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260808 23:45:34.493193 11550 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
I20260808 23:45:34.494922 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.494411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53560)
0808 23:45:34.494630 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.494744 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 23:45:34.495224 11605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.186.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab.
Entry for principal kudu/127.18.186.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab.
I20260808 23:45:34.519860 19178 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
/tmp/dist-test-taskX1Oigb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.188:38925
--webserver_interface=127.18.186.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab
--principal=kudu/127.18.186.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.186.148:41557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.186.190:42181,127.18.186.189:32959,127.18.186.188:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:45:34.544718 11605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 1
I20260808 23:45:34.546021 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.545557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53576)
0808 23:45:34.545758 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.545863 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 23:45:34.546988 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.546653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53590)
0808 23:45:34.546816 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.546888 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 23:45:34.547188 11605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
I20260808 23:45:34.615483 11605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 2
I20260808 23:45:34.616722 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.616231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53598)
0808 23:45:34.616456 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.616560 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 23:45:34.617816 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.617510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53608)
0808 23:45:34.617644 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.617709 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 23:45:34.618079 11605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
I20260808 23:45:34.620994 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } attempt: 5
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:45:34.635915 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.621632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:57894)
0808 23:45:34.621809 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.621834 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:34.621940 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:34.622314 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:34.622317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:34.622697 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:34.622874 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.622883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.624310 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.624315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.625210 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.625222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.626319 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.626323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.626492 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.627145 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:34.627166 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:34.629585 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:34.633325 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:34.633329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:34.633332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:34.633728 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:34.634169 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:34.634173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:34.634175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:34.634350 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:34.634967 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:34.634970 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:34.635122 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 23:45:34.635259 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 23:45:34.635618 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 23:45:34.636021 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.621684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.18.186.189:32959)
0808 23:45:34.621862 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:34.621868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:34.621892 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:45:34.622061 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:34.622066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:34.622166 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:34.622277 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:34.623022 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.624165 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.625378 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.626184 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.626308 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.629766 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:34.629772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:34.629777 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:34.629818 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:34.633169 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.633856 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.633862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.633866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.634023 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.634470 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.634474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.634478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.634836 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:34.634984 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:34.635445 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 23:45:34.635721 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:45:34.637182 11525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:45:34.638329 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.637926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53624)
0808 23:45:34.638105 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.638190 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 23:45:34.639338 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.638992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53628)
0808 23:45:34.639152 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.639230 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:45:34.639578 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
I20260808 23:45:34.690842 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 1
I20260808 23:45:34.692166 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.691643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53642)
0808 23:45:34.691859 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.691981 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 23:45:34.693378 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.693038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53654)
0808 23:45:34.693184 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.693255 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 23:45:34.693640 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
I20260808 23:45:34.709328 11605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 3
I20260808 23:45:34.710407 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.709953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53656)
0808 23:45:34.710139 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.710237 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 23:45:34.711550 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.711083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53668)
0808 23:45:34.711306 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.711382 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260808 23:45:34.711761 11605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
I20260808 23:45:34.761879 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 2
I20260808 23:45:34.763172 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.762653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53678)
0808 23:45:34.762907 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.763005 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 23:45:34.764344 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.763948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53692)
0808 23:45:34.764112 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.764207 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 23:45:34.764612 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
I20260808 23:45:34.854990 11605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 4
I20260808 23:45:34.856166 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.855680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53704)
0808 23:45:34.855917 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.856000 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 23:45:34.857214 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.856886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53716)
0808 23:45:34.857046 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.857113 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 23:45:34.857508 11605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
W20260808 23:45:34.862162 11614 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:45:34.862502 11614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:45:34.862615 11614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:45:34.862746 11614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 23:45:34.870849 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 3
W20260808 23:45:34.871249 11614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:45:34.871361 11614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:45:34.871500 11614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:45:34.871589 11614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:45:34.871691 11614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:45:34.872234 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.871571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53724)
0808 23:45:34.871892 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.872015 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 23:45:34.873605 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.873042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53736)
0808 23:45:34.873323 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.873441 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260808 23:45:34.873869 11525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.186.188:38925: Network error: Client connection negotiation failed: client connection to 127.18.186.188:38925: connect: Connection refused (error 111)
I20260808 23:45:34.883155 11614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.186.148:41557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.186.190:42181,127.18.186.189:32959,127.18.186.188:38925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/krb5kdc/kudu_127.18.186.188.keytab
--principal=kudu/127.18.186.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.186.188:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.186.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:45:34.884442 11614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:45:34.885816 11614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:45:34.897161 11620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:34.897182 11623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:45:34.897286 11621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:45:34.898720 11614 server_base.cc:1034] running on GCE node
I20260808 23:45:34.899494 11614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:45:34.900898 11614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:45:34.902158 11614 hybrid_clock.cc:648] HybridClock initialized: now 1786232734902064 us; error 56 us; skew 500 ppm
I20260808 23:45:34.902486 11614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:45:34.905629 11614 init.cc:377] Logged in from keytab as kudu/127.18.186.188@KRBTEST.COM (short username kudu)
I20260808 23:45:34.907052 11614 webserver.cc:460] Webserver started at http://127.18.186.188:46651/ using document root <none> and password file <none>
I20260808 23:45:34.907771 11614 fs_manager.cc:359] Metadata directory not provided
I20260808 23:45:34.907871 11614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:45:34.908118 11614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:45:34.910938 11614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/data/instance:
uuid: "302c1904d99847bca8ea28e3e0d83eb9"
format_stamp: "Formatted at 2026-08-08 23:45:34 on dist-test-slave-5k54"
server_key: "b546c80f7c930c7930836cba2a512c65"
server_key_iv: "ac81a3ba2aee94ac9ca5d01680f27987"
server_key_version: "encryptionkey@0"
I20260808 23:45:34.911734 11614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/wal/instance:
uuid: "302c1904d99847bca8ea28e3e0d83eb9"
format_stamp: "Formatted at 2026-08-08 23:45:34 on dist-test-slave-5k54"
server_key: "b546c80f7c930c7930836cba2a512c65"
server_key_iv: "ac81a3ba2aee94ac9ca5d01680f27987"
server_key_version: "encryptionkey@0"
I20260808 23:45:34.916781 11614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:45:34.920285 11630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:45:34.921778 11614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:45:34.921955 11614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/data,/tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/wal
uuid: "302c1904d99847bca8ea28e3e0d83eb9"
format_stamp: "Formatted at 2026-08-08 23:45:34 on dist-test-slave-5k54"
server_key: "b546c80f7c930c7930836cba2a512c65"
server_key_iv: "ac81a3ba2aee94ac9ca5d01680f27987"
server_key_version: "encryptionkey@0"
I20260808 23:45:34.922173 11614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:45:34.935063 11614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:45:34.938812 11614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:45:34.939073 11614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:45:34.957409 11614 rpc_server.cc:307] RPC server started. Bound to: 127.18.186.188:38925
I20260808 23:45:34.957441 11685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.186.188:38925 every 8 connection(s)
I20260808 23:45:34.958716 11614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/data/info.pb
I20260808 23:45:34.961807 19178 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu as pid 11614
I20260808 23:45:34.962005 19178 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskX1Oigb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786232366675349-19178-0/minicluster-data/master-2/wal/instance
I20260808 23:45:34.962448 11686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:45:34.962466 19178 external_mini_cluster.cc:1428] Setting key 9f6ce22556b926531aa94690007b064f
I20260808 23:45:34.972147 11686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:34.987249 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.964217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53746 (local address 127.18.186.188:38925)
0808 23:45:34.964993 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:34.965004 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:34.965046 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:45:34.967169 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:34.967190 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:34.967655 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:34.968004 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:34.969803 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.971076 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.972909 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.973725 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.973855 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.978779 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:34.978790 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:34.978805 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:34.978869 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:34.982889 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.983600 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.983610 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.983622 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.983759 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.984270 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.984275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.984278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.985071 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:34.985311 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:34.985747 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 23:45:34.986177 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":352,"threads_started":1}
I20260808 23:45:34.990911 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.975172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.190:42181 (local address 127.0.0.1:35272)
0808 23:45:34.976032 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.976071 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:34.976232 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:34.976617 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:34.976624 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:34.977347 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:34.977698 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.977712 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.978825 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.978831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.980609 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.980630 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.981457 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.981470 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.981655 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.982244 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:34.982271 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:34.985021 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:34.988303 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:34.988315 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:34.988322 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:34.988736 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:34.989178 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:34.989181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:34.989183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:34.989298 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:34.989866 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:34.989872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:34.990066 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 23:45:34.990270 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 23:45:34.990578 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":515,"threads_started":1}
I20260808 23:45:34.991278 11690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.975114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.18.186.190:42181)
0808 23:45:34.975738 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:34.975746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:34.976161 (+ 415us) server_negotiation.cc:407] Connection header received
0808 23:45:34.976375 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:34.976379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:34.976443 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:34.976552 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:34.977895 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.978674 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.980766 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.981260 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.981354 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.985244 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:34.985250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:34.985254 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:34.985295 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:34.988181 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.988906 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.988911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.988914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.989066 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:34.989422 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:34.989426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:34.989428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:34.989729 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:34.989882 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:34.990416 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 23:45:34.990713 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":343,"threads_started":1}
I20260808 23:45:34.992882 11686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
Aug 08 23:45:34 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.190@KRBTEST.COM
I20260808 23:45:35.007251 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.993652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.18.186.189:32959)
0808 23:45:34.993826 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:34.993830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:34.993845 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:45:34.993891 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:34.993894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:34.993948 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:34.994024 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:34.994821 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.995893 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.997489 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.997956 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.998058 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.001504 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.001510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.001516 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.001557 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.004301 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.005051 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.005054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.005057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.005181 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.005622 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.005626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.005629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.006041 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.006220 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.006703 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.006952 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:45:35.007241 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.993469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.189:32959 (local address 127.0.0.1:57908)
0808 23:45:34.993652 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:34.993671 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:34.993759 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:34.994078 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:34.994083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:34.994527 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:34.994683 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.994691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.996044 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.996047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.997347 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:34.997364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.998120 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.998126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:34.998339 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:34.998864 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:34.998891 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:35.001316 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:35.004451 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:35.004456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:35.004460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:35.004941 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:35.005287 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:35.005294 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:35.005299 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:35.005480 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:35.006205 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:35.006208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:35.006417 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 23:45:35.006592 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 23:45:35.006885 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 23:45:35.008374 11686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:45:35.008677 11690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:34.990304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35284 (local address 127.18.186.190:42181)
0808 23:45:34.991565 (+ 1261us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:34.991570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:34.991591 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:45:34.991642 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:34.991648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:34.991753 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:34.991867 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:34.993376 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.994411 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.996118 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:34.996776 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:34.996946 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.002372 (+ 5426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.002379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.002385 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.002428 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.005663 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.006447 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.006453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.006457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.006615 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.007045 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.007052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.007056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.007484 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.007747 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.008161 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.008375 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1153,"thread_start_us":1151,"threads_started":1}
Aug 08 23:45:35 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.188@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
Aug 08 23:45:35 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.186.189@KRBTEST.COM
I20260808 23:45:35.022598 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.009359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.18.186.188:38925)
0808 23:45:35.009538 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.009544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.009563 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:45:35.009639 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.009644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.009721 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.009822 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.010804 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.011752 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.013139 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.013756 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.013850 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.017127 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.017131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.017135 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.017161 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.019966 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.020725 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.020729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.020731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.020873 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.021299 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.021303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.021305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.021573 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.021679 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.022151 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.022348 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":70}
I20260808 23:45:35.023088 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.009274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53750)
0808 23:45:35.009450 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:35.009469 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:35.009560 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:35.009916 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:35.009921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:35.010433 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:35.010631 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:35.010643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.011890 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.011897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:35.013000 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:35.013012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.013928 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.013935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:35.014152 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.014734 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:35.014759 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:35.016953 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:35.020090 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:35.020098 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:35.020101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:35.020607 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:35.020990 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:35.020995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:35.020999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:35.021184 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:35.021693 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:35.021700 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:35.021863 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 23:45:35.022062 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 23:45:35.022510 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":123}
I20260808 23:45:35.026566 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.010674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.18.186.189:32959)
0808 23:45:35.010836 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.010843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.010863 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:45:35.010986 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.010991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.011060 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.011154 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.012527 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.013435 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.015162 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.015620 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.015712 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.020298 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.020304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.020308 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.020345 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.023779 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.024470 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.024475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.024478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.024644 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.025057 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.025063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.025066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.025441 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.025705 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.025974 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.026207 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:45:35.029389 11686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: Bootstrap starting.
I20260808 23:45:35.031459 19178 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:45:35.032868 11686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:35.033895 11686 log.cc:824] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:35.033774 11499 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35284
I20260808 23:45:35.036580 11686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: No bootstrap required, opened a new log
I20260808 23:45:35.043538 11686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } }
I20260808 23:45:35.043883 11686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302c1904d99847bca8ea28e3e0d83eb9, State: Initialized, Role: FOLLOWER
I20260808 23:45:35.044292 11686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } }
I20260808 23:45:35.045288 11694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.045663 11694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:35.046392 11686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:35.051119 11525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 4
Aug 08 23:45:35 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232733, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.190@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
W20260808 23:45:35.066207 11705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:45:35.066481 11705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:45:35.053458 11614 master_runner.cc:186] Error getting master registration for 127.18.186.190:42181: OK, instance_id { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" instance_seqno: 1786232733909994 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:45:35.067354 11614 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:45:35.070785 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.051850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53754)
0808 23:45:35.052033 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:35.052056 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:35.052230 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:35.052786 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:35.052791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:35.053652 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:35.053932 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:35.053950 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.055609 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.055617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:35.057566 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:35.057586 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.058789 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.058798 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:35.059115 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.059956 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:35.059988 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:35.063215 (+ 3227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:35.068018 (+ 4803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:35.068025 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:35.068031 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:35.068543 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:35.068977 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:35.068981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:35.068985 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:35.069160 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:35.069766 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:35.069771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:35.069944 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 23:45:35.070099 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 23:45:35.070380 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 23:45:35.070868 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.052158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.18.186.188:38925)
0808 23:45:35.052412 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.052421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.052443 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:45:35.052524 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.052529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.052605 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.052706 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.054105 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.055372 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.057757 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.058612 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.058745 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.063486 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.063494 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.063501 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.063547 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.067872 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.068677 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.068684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.068688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.068858 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.069276 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.069281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.069284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.069649 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.069812 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.070247 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.070472 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:45:35.075790 11525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: Bootstrap starting.
I20260808 23:45:35.078998 11525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:35.080408 11525 log.cc:824] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:35.083817 11525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: No bootstrap required, opened a new log
I20260808 23:45:35.091413 11525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } }
I20260808 23:45:35.091873 11525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a9c5f6a0aa4d9f98f7b3d88bce4ff7, State: Initialized, Role: FOLLOWER
I20260808 23:45:35.092366 11525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } }
I20260808 23:45:35.093323 11708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.093766 11708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:35.094547 11525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:35.094681 11499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 23:45:35.094906 11499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:35.099109 11499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } }
I20260808 23:45:35.101948 11499 leader_election.cc:288] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [CANDIDATE]: Term 1 election: Requested vote from peers b72a6042aa994fba88a9ce710ebbcb4b (127.18.186.189:32959), 302c1904d99847bca8ea28e3e0d83eb9 (127.18.186.188:38925)
I20260808 23:45:35.102965 11660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "302c1904d99847bca8ea28e3e0d83eb9"
I20260808 23:45:35.103499 11660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:35.107187 11660 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 in term 1.
I20260808 23:45:35.109912 11472 leader_election.cc:302] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302c1904d99847bca8ea28e3e0d83eb9, 97a9c5f6a0aa4d9f98f7b3d88bce4ff7; no voters:
I20260808 23:45:35.110517 11708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [term 1 FOLLOWER]: Leader election won for term 1
W20260808 23:45:35.126332 11720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:45:35.126554 11720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:45:35.134660 11708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [term 1 LEADER]: Becoming Leader. State: Replica: 97a9c5f6a0aa4d9f98f7b3d88bce4ff7, State: Running, Role: LEADER
I20260808 23:45:35.135416 11708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } }
I20260808 23:45:35.104707 11456 master_runner.cc:186] Error getting master registration for 127.18.186.189:32959: OK, instance_id { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" instance_seqno: 1786232734426305 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:45:35.138778 11456 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:45:35.139997 11723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97a9c5f6a0aa4d9f98f7b3d88bce4ff7. Latest consensus state: current_term: 1 leader_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.140463 11723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:35.141671 11729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:45:35.144382 11729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 23:45:35.149114 11477 tablet.cc:2369] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:45:35.156714 11605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } attempt: 5
I20260808 23:45:35.177438 11660 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [term 1 FOLLOWER]: Refusing update from remote peer 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:45:35.178536 11708 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 08 23:45:35 dist-test-slave-5k54 krb5kdc[11442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786232734, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.186.189@KRBTEST.COM for kudu/127.18.186.188@KRBTEST.COM
I20260808 23:45:35.182308 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.149361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57918 (local address 127.18.186.189:32959)
0808 23:45:35.149555 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.149561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.151364 (+ 1803us) server_negotiation.cc:407] Connection header received
0808 23:45:35.151554 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.151559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.151635 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.151728 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.166057 (+ 14329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.167172 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.169271 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.169927 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.170022 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.173518 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.173524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.173528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.173575 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.177627 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.178461 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.178468 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.178472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.178645 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.179106 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.179112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.179117 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.179541 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.179792 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.181159 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.181816 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:45:35.190529 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.137206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.18.186.188:38925)
0808 23:45:35.137404 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.137410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.150318 (+ 12908us) server_negotiation.cc:407] Connection header received
0808 23:45:35.151868 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.151877 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.152006 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.152097 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.155068 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.156310 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.162820 (+ 6510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.163542 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.163639 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.169589 (+ 5950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.169596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.169601 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.169641 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.173548 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.185599 (+ 12051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.185605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.185609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.185779 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.187340 (+ 1561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.187346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.187350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.187759 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.188032 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.189874 (+ 1842us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.190131 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 23:45:35.191099 11637 tablet.cc:2369] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:45:35.194298 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.157305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.186.188:38925 (local address 127.0.0.1:53772)
0808 23:45:35.157660 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 23:45:35.157679 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:45:35.157777 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:45:35.171960 (+ 14183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:45:35.171966 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:45:35.172468 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:45:35.172691 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:35.172703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.174881 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.174887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:35.176452 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:45:35.176469 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.177533 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.177539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:45:35.177775 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.178586 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:45:35.178614 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:45:35.181633 (+ 3019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:45:35.189763 (+ 8130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:35.189769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:35.189773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:35.190343 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:35.191529 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:45:35.191533 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:45:35.191536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:45:35.191760 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:45:35.192762 (+ 1002us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:45:35.192767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:45:35.192937 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 23:45:35.193085 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:45:35.193406 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
I20260808 23:45:35.196373 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.157974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53772 (local address 127.18.186.188:38925)
0808 23:45:35.168958 (+ 10984us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.168963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.168987 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:45:35.169036 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.169041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.169124 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.172953 (+ 3829us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.173246 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.174498 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.176668 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.177346 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.181435 (+ 4089us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.181902 (+ 467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.181912 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.181916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.181984 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.189535 (+ 7551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.190572 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.190866 (+ 294us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.190872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.191305 (+ 433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.191969 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.191973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.192002 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.192562 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.195033 (+ 2471us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.195244 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.195667 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10841,"thread_start_us":11229,"threads_started":1}
I20260808 23:45:35.202661 11605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b: Bootstrap starting.
I20260808 23:45:35.204533 11690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:35.140469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35296 (local address 127.18.186.190:42181)
0808 23:45:35.140700 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.140711 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.150990 (+ 10279us) server_negotiation.cc:407] Connection header received
0808 23:45:35.151450 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.151458 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.151545 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.151621 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.161161 (+ 9540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.162339 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.191519 (+ 29180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:35.192327 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:35.192452 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:35.196832 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:45:35.196843 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:45:35.196851 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:45:35.196925 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:45:35.200982 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.201846 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.201852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.201855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.202013 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:45:35.202493 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:45:35.202499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:45:35.202501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:45:35.202893 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:45:35.203165 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:35.203850 (+ 685us) server_negotiation.cc:299] Negotiation successful
0808 23:45:35.204104 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:45:35.211144 11605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b: Neither blocks nor log segments found. Creating new log.
I20260808 23:45:35.213294 11605 log.cc:824] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:45:35.215843 11694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97a9c5f6a0aa4d9f98f7b3d88bce4ff7. Latest consensus state: current_term: 1 leader_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.216166 11694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:35.220289 11732 mvcc.cc:205] Tried to move back new op lower bound from 7316409283258019840 to 7316409283120463872. Current Snapshot: MvccSnapshot[applied={T|T < 7316409283258019840}]
I20260808 23:45:35.222738 11605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b: No bootstrap required, opened a new log
I20260808 23:45:35.223862 11708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.224159 11708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:35.225541 11694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.225827 11694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:35.226073 11735 mvcc.cc:205] Tried to move back new op lower bound from 7316409283258019840 to 7316409283120463872. Current Snapshot: MvccSnapshot[applied={T|T < 7316409283258019840}]
I20260808 23:45:35.236073 11729 catalog_manager.cc:1347] Generated new cluster ID: 173a705139144cd3bd73f9bf5cec8d83
I20260808 23:45:35.236186 11729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:45:35.252736 11605 raft_consensus.cc:348] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } }
I20260808 23:45:35.253257 11605 raft_consensus.cc:729] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b72a6042aa994fba88a9ce710ebbcb4b, State: Initialized, Role: FOLLOWER
I20260808 23:45:35.253976 11605 consensus_queue.cc:260] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } }
I20260808 23:45:35.255344 11740 sys_catalog.cc:455] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.256117 11740 sys_catalog.cc:458] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:35.263574 11605 sys_catalog.cc:564] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:45:35.263618 11580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b72a6042aa994fba88a9ce710ebbcb4b"
I20260808 23:45:35.264127 11580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:45:35.267704 11580 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 in term 1.
I20260808 23:45:35.269217 11579 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [term 1 FOLLOWER]: Refusing update from remote peer 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:45:35.270196 11723 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:45:35.282133 11729 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:45:35.284123 11729 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:45:35.297473 11729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7: Generated new TSK 0
I20260808 23:45:35.298501 11729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:45:35.333896 11723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.334267 11723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a9c5f6a0aa4d9f98f7b3d88bce4ff7 [sys.catalog]: This master's current role is: LEADER
I20260808 23:45:35.339231 11740 sys_catalog.cc:455] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97a9c5f6a0aa4d9f98f7b3d88bce4ff7. Latest consensus state: current_term: 1 leader_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.339613 11740 sys_catalog.cc:458] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:35.339972 11744 sys_catalog.cc:455] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a9c5f6a0aa4d9f98f7b3d88bce4ff7" member_type: VOTER last_known_addr { host: "127.18.186.190" port: 42181 } } peers { permanent_uuid: "b72a6042aa994fba88a9ce710ebbcb4b" member_type: VOTER last_known_addr { host: "127.18.186.189" port: 32959 } } peers { permanent_uuid: "302c1904d99847bca8ea28e3e0d83eb9" member_type: VOTER last_known_addr { host: "127.18.186.188" port: 38925 } } }
I20260808 23:45:35.340358 11744 sys_catalog.cc:458] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:45:35.377522 11758 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260808 23:45:35.431222 11690 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.414607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.18.186.190:42181)
0808 23:45:35.415625 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.415631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.428142 (+ 12511us) server_negotiation.cc:407] Connection header received
0808 23:45:35.429598 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.429605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.429691 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.429768 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.430970 (+ 1202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35302: BlockingRecv error: recv got EOF from 127.0.0.1:35302 (error 108)
Metrics: {"server-negotiator.queue_time_us":858}
W20260808 23:45:35.433954 11734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.414671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.18.186.188:38925)
0808 23:45:35.414862 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.414869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.432017 (+ 17148us) server_negotiation.cc:407] Connection header received
0808 23:45:35.432284 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.432292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.432409 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.432484 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.433728 (+ 1244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53788: BlockingRecv error: recv got EOF from 127.0.0.1:53788 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 23:45:35.434041 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.428717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57930 (local address 127.18.186.189:32959)
0808 23:45:35.428907 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.428912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.432166 (+ 3254us) server_negotiation.cc:407] Connection header received
0808 23:45:35.432387 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.432392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.432489 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.432553 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.433844 (+ 1291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57930: BlockingRecv error: recv got EOF from 127.0.0.1:57930 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 23:45:35.480540 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.477941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.18.186.189:32959)
0808 23:45:35.478178 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.478188 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.478782 (+ 594us) server_negotiation.cc:407] Connection header received
0808 23:45:35.479089 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.479095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.479219 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.479362 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.480244 (+ 882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57942: BlockingRecv error: recv got EOF from 127.0.0.1:57942 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 23:45:35.489494 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.484977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.18.186.189:32959)
0808 23:45:35.485201 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.485210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.488004 (+ 2794us) server_negotiation.cc:407] Connection header received
0808 23:45:35.488227 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.488240 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.488319 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.488437 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.489283 (+ 846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57944: BlockingRecv error: recv got EOF from 127.0.0.1:57944 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 23:45:35.507288 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.505523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.18.186.189:32959)
0808 23:45:35.505690 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.505695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.505801 (+ 106us) server_negotiation.cc:407] Connection header received
0808 23:45:35.506086 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.506092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.506187 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.506296 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.507048 (+ 752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57950: BlockingRecv error: recv got EOF from 127.0.0.1:57950 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 23:45:35.516736 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.514556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57960 (local address 127.18.186.189:32959)
0808 23:45:35.514820 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.514829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.515331 (+ 502us) server_negotiation.cc:407] Connection header received
0808 23:45:35.515572 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.515577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.515673 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.515782 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.516512 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57960: BlockingRecv error: recv got EOF from 127.0.0.1:57960 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260808 23:45:35.530972 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.528747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.18.186.189:32959)
0808 23:45:35.528979 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.528988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.529488 (+ 500us) server_negotiation.cc:407] Connection header received
0808 23:45:35.529715 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.529719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.529799 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.529911 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.530730 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57962: BlockingRecv error: recv got EOF from 127.0.0.1:57962 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 23:45:35.553659 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.550976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.18.186.189:32959)
0808 23:45:35.551245 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.551255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.552005 (+ 750us) server_negotiation.cc:407] Connection header received
0808 23:45:35.552275 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.552281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.552364 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.552472 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.553421 (+ 949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57970: BlockingRecv error: recv got EOF from 127.0.0.1:57970 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 23:45:35.592285 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.589758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57980 (local address 127.18.186.189:32959)
0808 23:45:35.589972 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.589979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.590595 (+ 616us) server_negotiation.cc:407] Connection header received
0808 23:45:35.590826 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.590834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.590936 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.591062 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.592010 (+ 948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57980: BlockingRecv error: recv got EOF from 127.0.0.1:57980 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 23:45:35.661993 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.660060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.18.186.189:32959)
0808 23:45:35.660304 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.660311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.660699 (+ 388us) server_negotiation.cc:407] Connection header received
0808 23:45:35.660867 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.660872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.660969 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.661095 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.661786 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57988: BlockingRecv error: recv got EOF from 127.0.0.1:57988 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 23:45:35.795912 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:35.793710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.18.186.189:32959)
0808 23:45:35.793924 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:35.793929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:35.794305 (+ 376us) server_negotiation.cc:407] Connection header received
0808 23:45:35.794501 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:35.794510 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:35.794616 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:35.794725 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:35.795577 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57992: BlockingRecv error: recv got EOF from 127.0.0.1:57992 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 23:45:36.057690 11610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:45:36.055995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.18.186.189:32959)
0808 23:45:36.056180 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:36.056185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:36.056483 (+ 298us) server_negotiation.cc:407] Connection header received
0808 23:45:36.056660 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:36.056664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:36.056738 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:36.056833 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:45:36.057525 (+ 692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57998: BlockingRecv error: recv got EOF from 127.0.0.1:57998 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:45:36.068543 11705 catalog_manager.cc:1259] Loaded cluster ID: 173a705139144cd3bd73f9bf5cec8d83
I20260808 23:45:36.068645 11705 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: loading cluster ID for follower catalog manager: success
I20260808 23:45:36.071028 11705 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: acquiring CA information for follower catalog manager: success
I20260808 23:45:36.072851 11705 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 302c1904d99847bca8ea28e3e0d83eb9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:45:36.381836 11758 catalog_manager.cc:1259] Loaded cluster ID: 173a705139144cd3bd73f9bf5cec8d83
I20260808 23:45:36.381968 11758 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b: loading cluster ID for follower catalog manager: success
I20260808 23:45:36.384616 11758 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b: acquiring CA information for follower catalog manager: success
I20260808 23:45:36.386445 11758 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b72a6042aa994fba88a9ce710ebbcb4b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:45:36.581720 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:36.574091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.18.186.189:32959)
0808 23:45:36.574989 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:36.574996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:36.575021 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:45:36.575112 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:36.575119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:36.575510 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:36.575667 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:36.576171 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.577527 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.578837 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.579561 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.579690 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:36.580223 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:36.580971 (+ 748us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:36.581141 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:36.581245 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 23:45:36.581336 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":554,"threads_started":1}
I20260808 23:45:36.595283 11832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:36.588369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.18.186.188:38925)
0808 23:45:36.589072 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:36.589078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:36.589100 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:45:36.589214 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:36.589220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:36.589434 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:36.589612 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:36.589953 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.591121 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.592558 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.593230 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.593361 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:36.593831 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:36.594603 (+ 772us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:36.594750 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:36.594803 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 23:45:36.594902 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":446,"threads_started":1}
I20260808 23:45:36.623746 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:36.602098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35316 (local address 127.18.186.190:42181)
0808 23:45:36.614407 (+ 12309us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:36.614417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:36.615185 (+ 768us) server_negotiation.cc:407] Connection header received
0808 23:45:36.615536 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:36.615542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:36.615837 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:36.616037 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:36.616846 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.617990 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.620206 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.620877 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.621006 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:36.621848 (+ 842us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:36.622305 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:36.622419 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:36.623265 (+ 846us) server_negotiation.cc:299] Negotiation successful
0808 23:45:36.623366 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12105,"thread_start_us":13307,"threads_started":1}
I20260808 23:45:36.624835 11832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:36.602735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53800 (local address 127.18.186.188:38925)
0808 23:45:36.608437 (+ 5702us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:36.608446 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:36.616128 (+ 7682us) server_negotiation.cc:407] Connection header received
0808 23:45:36.616429 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:36.616435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:36.616542 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:36.616611 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:36.617579 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.619022 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.621938 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.622858 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.622967 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:36.623646 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:36.623790 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:36.623859 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:36.624445 (+ 586us) server_negotiation.cc:299] Negotiation successful
0808 23:45:36.624538 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5560}
I20260808 23:45:36.629103 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:45:36.616293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.18.186.189:32959)
0808 23:45:36.616492 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 23:45:36.616497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:45:36.623054 (+ 6557us) server_negotiation.cc:407] Connection header received
0808 23:45:36.623218 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:45:36.623224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:45:36.623327 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:45:36.623457 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:45:36.623990 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.625083 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.626710 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:45:36.627407 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:45:36.627577 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:45:36.628202 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:45:36.628343 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:45:36.628442 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:45:36.628672 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 23:45:36.628772 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:45:36.633584 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 11456
I20260808 23:45:36.634871 11520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:36.747596 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 11456
I20260808 23:45:36.773423 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 11534
I20260808 23:45:36.774544 11600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:36.888203 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 11534
I20260808 23:45:36.911630 19178 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu pid 11614
I20260808 23:45:36.912817 11681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:45:37.020987 19178 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskX1Oigb/build/asan/bin/kudu with pid 11614
2026-08-08T23:45:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5069 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9727 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (370276 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST